This is /home/cyd/emacs/doc/lispref/../../info/elisp, produced by
makeinfo version 4.13 from /home/cyd/emacs/doc/lispref/elisp.texi.

This is the `GNU Emacs Lisp Reference Manual' corresponding to Emacs
version 24.2.

Copyright (C) 1990-1996, 1998-2012 Free Software Foundation, Inc.

     Permission is granted to copy, distribute and/or modify this
     document under the terms of the GNU Free Documentation License,
     Version 1.3 or any later version published by the Free Software
     Foundation; with the Invariant Sections being "GNU General Public
     License," with the Front-Cover texts being "A GNU Manual," and
     with the Back-Cover Texts as in (a) below.  A copy of the license
     is included in the section entitled "GNU Free Documentation
     License."

     (a) The FSF's Back-Cover Text is: "You have the freedom to copy and
     modify this GNU manual.  Buying copies from the FSF supports it in
     developing GNU and promoting software freedom."

INFO-DIR-SECTION GNU Emacs Lisp
START-INFO-DIR-ENTRY
* Elisp: (elisp).       The Emacs Lisp Reference Manual.
END-INFO-DIR-ENTRY


File: elisp,  Node: Character Properties,  Next: Character Sets,  Prev: Character Codes,  Up: Non-ASCII Characters

33.5 Character Properties
=========================

A "character property" is a named attribute of a character that
specifies how the character behaves and how it should be handled during
text processing and display.  Thus, character properties are an
important part of specifying the character's semantics.

   On the whole, Emacs follows the Unicode Standard in its
implementation of character properties.  In particular, Emacs supports
the Unicode Character Property Model
(http://www.unicode.org/reports/tr23/), and the Emacs character
property database is derived from the Unicode Character Database (UCD).
See the Character Properties chapter of the Unicode Standard
(http://www.unicode.org/versions/Unicode5.0.0/ch04.pdf), for a detailed
description of Unicode character properties and their meaning.  This
section assumes you are already familiar with that chapter of the
Unicode Standard, and want to apply that knowledge to Emacs Lisp
programs.

   In Emacs, each property has a name, which is a symbol, and a set of
possible values, whose types depend on the property; if a character
does not have a certain property, the value is `nil'.  As a general
rule, the names of character properties in Emacs are produced from the
corresponding Unicode properties by downcasing them and replacing each
`_' character with a dash `-'.  For example,
`Canonical_Combining_Class' becomes `canonical-combining-class'.
However, sometimes we shorten the names to make their use easier.

   Some codepoints are left "unassigned" by the UCD--they don't
correspond to any character.  The Unicode Standard defines default
values of properties for such codepoints; they are mentioned below for
each property.

   Here is the full list of value types for all the character
properties that Emacs knows about:

`name'
     Corresponds to the `Name' Unicode property.  The value is a string
     consisting of upper-case Latin letters A to Z, digits, spaces, and
     hyphen `-' characters.  For unassigned codepoints, the value is an
     empty string.

`general-category'
     Corresponds to the `General_Category' Unicode property.  The value
     is a symbol whose name is a 2-letter abbreviation of the
     character's classification.  For unassigned codepoints, the value
     is `Cn'.

`canonical-combining-class'
     Corresponds to the `Canonical_Combining_Class' Unicode property.
     The value is an integer number.  For unassigned codepoints, the
     value is zero.

`bidi-class'
     Corresponds to the Unicode `Bidi_Class' property.  The value is a
     symbol whose name is the Unicode "directional type" of the
     character.  Emacs uses this property when it reorders bidirectional
     text for display (*note Bidirectional Display::).  For unassigned
     codepoints, the value depends on the code blocks to which the
     codepoint belongs: most unassigned codepoints get the value of `L'
     (strong L), but some get values of `AL' (Arabic letter) or `R'
     (strong R).

`decomposition'
     Corresponds to the Unicode properties `Decomposition_Type' and
     `Decomposition_Value'.  The value is a list, whose first element
     may be a symbol representing a compatibility formatting tag, such
     as `small'(1); the other elements are characters that give the
     compatibility decomposition sequence of this character.  For
     unassigned codepoints, the value is the character itself.

`decimal-digit-value'
     Corresponds to the Unicode `Numeric_Value' property for characters
     whose `Numeric_Type' is `Digit'.  The value is an integer number.
     For unassigned codepoints, the value is `nil', which means NaN, or
     "not-a-number".

`digit-value'
     Corresponds to the Unicode `Numeric_Value' property for characters
     whose `Numeric_Type' is `Decimal'.  The value is an integer
     number.  Examples of such characters include compatibility
     subscript and superscript digits, for which the value is the
     corresponding number.  For unassigned codepoints, the value is
     `nil', which means NaN.

`numeric-value'
     Corresponds to the Unicode `Numeric_Value' property for characters
     whose `Numeric_Type' is `Numeric'.  The value of this property is
     an integer or a floating-point number.  Examples of characters
     that have this property include fractions, subscripts,
     superscripts, Roman numerals, currency numerators, and encircled
     numbers.  For example, the value of this property for the character
     `U+2155' (VULGAR FRACTION ONE FIFTH) is `0.2'.  For unassigned
     codepoints, the value is `nil', which means NaN.

`mirrored'
     Corresponds to the Unicode `Bidi_Mirrored' property.  The value of
     this property is a symbol, either `Y' or `N'.  For unassigned
     codepoints, the value is `N'.

`mirroring'
     Corresponds to the Unicode `Bidi_Mirroring_Glyph' property.  The
     value of this property is a character whose glyph represents the
     mirror image of the character's glyph, or `nil' if there's no
     defined mirroring glyph.  All the characters whose `mirrored'
     property is `N' have `nil' as their `mirroring' property; however,
     some characters whose `mirrored' property is `Y' also have `nil'
     for `mirroring', because no appropriate characters exist with
     mirrored glyphs.  Emacs uses this property to display mirror
     images of characters when appropriate (*note Bidirectional
     Display::).  For unassigned codepoints, the value is `nil'.

`old-name'
     Corresponds to the Unicode `Unicode_1_Name' property.  The value
     is a string.  For unassigned codepoints, the value is an empty
     string.

`iso-10646-comment'
     Corresponds to the Unicode `ISO_Comment' property.  The value is a
     string.  For unassigned codepoints, the value is an empty string.

`uppercase'
     Corresponds to the Unicode `Simple_Uppercase_Mapping' property.
     The value of this property is a single character.  For unassigned
     codepoints, the value is `nil', which means the character itself.

`lowercase'
     Corresponds to the Unicode `Simple_Lowercase_Mapping' property.
     The value of this property is a single character.  For unassigned
     codepoints, the value is `nil', which means the character itself.

`titlecase'
     Corresponds to the Unicode `Simple_Titlecase_Mapping' property.
     "Title case" is a special form of a character used when the first
     character of a word needs to be capitalized.  The value of this
     property is a single character.  For unassigned codepoints, the
     value is `nil', which means the character itself.

 -- Function: get-char-code-property char propname
     This function returns the value of CHAR's PROPNAME property.

          (get-char-code-property ?  'general-category)
               => Zs
          (get-char-code-property ?1  'general-category)
               => Nd
          ;; subscript 4
          (get-char-code-property ?\u2084 'digit-value)
               => 4
          ;; one fifth
          (get-char-code-property ?\u2155 'numeric-value)
               => 0.2
          ;; Roman IV
          (get-char-code-property ?\u2163 'numeric-value)
               => 4

 -- Function: char-code-property-description prop value
     This function returns the description string of property PROP's
     VALUE, or `nil' if VALUE has no description.

          (char-code-property-description 'general-category 'Zs)
               => "Separator, Space"
          (char-code-property-description 'general-category 'Nd)
               => "Number, Decimal Digit"
          (char-code-property-description 'numeric-value '1/5)
               => nil

 -- Function: put-char-code-property char propname value
     This function stores VALUE as the value of the property PROPNAME
     for the character CHAR.

 -- Variable: unicode-category-table
     The value of this variable is a char-table (*note Char-Tables::)
     that specifies, for each character, its Unicode `General_Category'
     property as a symbol.

 -- Variable: char-script-table
     The value of this variable is a char-table that specifies, for each
     character, a symbol whose name is the script to which the character
     belongs, according to the Unicode Standard classification of the
     Unicode code space into script-specific blocks.  This char-table
     has a single extra slot whose value is the list of all script
     symbols.

 -- Variable: char-width-table
     The value of this variable is a char-table that specifies the
     width of each character in columns that it will occupy on the
     screen.

 -- Variable: printable-chars
     The value of this variable is a char-table that specifies, for each
     character, whether it is printable or not.  That is, if evaluating
     `(aref printable-chars char)' results in `t', the character is
     printable, and if it results in `nil', it is not.

   ---------- Footnotes ----------

   (1) The Unicode specification writes these tag names inside `<..>'
brackets, but the tag names in Emacs do not include the brackets; e.g.
Unicode specifies `<small>' where Emacs uses `small'.


File: elisp,  Node: Character Sets,  Next: Scanning Charsets,  Prev: Character Properties,  Up: Non-ASCII Characters

33.6 Character Sets
===================

An Emacs "character set", or "charset", is a set of characters in which
each character is assigned a numeric code point.  (The Unicode Standard
calls this a "coded character set".)  Each Emacs charset has a name
which is a symbol.  A single character can belong to any number of
different character sets, but it will generally have a different code
point in each charset.  Examples of character sets include `ascii',
`iso-8859-1', `greek-iso8859-7', and `windows-1255'.  The code point
assigned to a character in a charset is usually different from its code
point used in Emacs buffers and strings.

   Emacs defines several special character sets.  The character set
`unicode' includes all the characters whose Emacs code points are in
the range `0..#x10FFFF'.  The character set `emacs' includes all ASCII
and non-ASCII characters.  Finally, the `eight-bit' charset includes
the 8-bit raw bytes; Emacs uses it to represent raw bytes encountered
in text.

 -- Function: charsetp object
     Returns `t' if OBJECT is a symbol that names a character set,
     `nil' otherwise.

 -- Variable: charset-list
     The value is a list of all defined character set names.

 -- Function: charset-priority-list &optional highestp
     This functions returns a list of all defined character sets
     ordered by their priority.  If HIGHESTP is non-`nil', the function
     returns a single character set of the highest priority.

 -- Function: set-charset-priority &rest charsets
     This function makes CHARSETS the highest priority character sets.

 -- Function: char-charset character &optional restriction
     This function returns the name of the character set of highest
     priority that CHARACTER belongs to.  ASCII characters are an
     exception: for them, this function always returns `ascii'.

     If RESTRICTION is non-`nil', it should be a list of charsets to
     search.  Alternatively, it can be a coding system, in which case
     the returned charset must be supported by that coding system
     (*note Coding Systems::).

 -- Function: charset-plist charset
     This function returns the property list of the character set
     CHARSET.  Although CHARSET is a symbol, this is not the same as
     the property list of that symbol.  Charset properties include
     important information about the charset, such as its documentation
     string, short name, etc.

 -- Function: put-charset-property charset propname value
     This function sets the PROPNAME property of CHARSET to the given
     VALUE.

 -- Function: get-charset-property charset propname
     This function returns the value of CHARSETs property PROPNAME.

 -- Command: list-charset-chars charset
     This command displays a list of characters in the character set
     CHARSET.

   Emacs can convert between its internal representation of a character
and the character's codepoint in a specific charset.  The following two
functions support these conversions.

 -- Function: decode-char charset code-point
     This function decodes a character that is assigned a CODE-POINT in
     CHARSET, to the corresponding Emacs character, and returns it.  If
     CHARSET doesn't contain a character of that code point, the value
     is `nil'.  If CODE-POINT doesn't fit in a Lisp integer (*note
     most-positive-fixnum: Integer Basics.), it can be specified as a
     cons cell `(HIGH . LOW)', where LOW are the lower 16 bits of the
     value and HIGH are the high 16 bits.

 -- Function: encode-char char charset
     This function returns the code point assigned to the character
     CHAR in CHARSET.  If the result does not fit in a Lisp integer, it
     is returned as a cons cell `(HIGH . LOW)' that fits the second
     argument of `decode-char' above.  If CHARSET doesn't have a
     codepoint for CHAR, the value is `nil'.

   The following function comes in handy for applying a certain
function to all or part of the characters in a charset:

 -- Function: map-charset-chars function charset &optional arg
          from-code to-code
     Call FUNCTION for characters in CHARSET.  FUNCTION is called with
     two arguments.  The first one is a cons cell `(FROM .  TO)', where
     FROM and TO indicate a range of characters contained in charset.
     The second argument passed to FUNCTION is ARG.

     By default, the range of codepoints passed to FUNCTION includes
     all the characters in CHARSET, but optional arguments FROM-CODE
     and TO-CODE limit that to the range of characters between these
     two codepoints of CHARSET.  If either of them is `nil', it
     defaults to the first or last codepoint of CHARSET, respectively.


File: elisp,  Node: Scanning Charsets,  Next: Translation of Characters,  Prev: Character Sets,  Up: Non-ASCII Characters

33.7 Scanning for Character Sets
================================

Sometimes it is useful to find out which character set a particular
character belongs to.  One use for this is in determining which coding
systems (*note Coding Systems::) are capable of representing all of the
text in question; another is to determine the font(s) for displaying
that text.

 -- Function: charset-after &optional pos
     This function returns the charset of highest priority containing
     the character at position POS in the current buffer.  If POS is
     omitted or `nil', it defaults to the current value of point.  If
     POS is out of range, the value is `nil'.

 -- Function: find-charset-region beg end &optional translation
     This function returns a list of the character sets of highest
     priority that contain characters in the current buffer between
     positions BEG and END.

     The optional argument TRANSLATION specifies a translation table to
     use for scanning the text (*note Translation of Characters::).  If
     it is non-`nil', then each character in the region is translated
     through this table, and the value returned describes the translated
     characters instead of the characters actually in the buffer.

 -- Function: find-charset-string string &optional translation
     This function returns a list of character sets of highest priority
     that contain characters in STRING.  It is just like
     `find-charset-region', except that it applies to the contents of
     STRING instead of part of the current buffer.


File: elisp,  Node: Translation of Characters,  Next: Coding Systems,  Prev: Scanning Charsets,  Up: Non-ASCII Characters

33.8 Translation of Characters
==============================

A "translation table" is a char-table (*note Char-Tables::) that
specifies a mapping of characters into characters.  These tables are
used in encoding and decoding, and for other purposes.  Some coding
systems specify their own particular translation tables; there are also
default translation tables which apply to all other coding systems.

   A translation table has two extra slots.  The first is either `nil'
or a translation table that performs the reverse translation; the
second is the maximum number of characters to look up for translating
sequences of characters (see the description of
`make-translation-table-from-alist' below).

 -- Function: make-translation-table &rest translations
     This function returns a translation table based on the argument
     TRANSLATIONS.  Each element of TRANSLATIONS should be a list of
     elements of the form `(FROM . TO)'; this says to translate the
     character FROM into TO.

     The arguments and the forms in each argument are processed in
     order, and if a previous form already translates TO to some other
     character, say TO-ALT, FROM is also translated to TO-ALT.

   During decoding, the translation table's translations are applied to
the characters that result from ordinary decoding.  If a coding system
has the property `:decode-translation-table', that specifies the
translation table to use, or a list of translation tables to apply in
sequence.  (This is a property of the coding system, as returned by
`coding-system-get', not a property of the symbol that is the coding
system's name.  *Note Basic Concepts of Coding Systems: Coding System
Basics.)  Finally, if `standard-translation-table-for-decode' is
non-`nil', the resulting characters are translated by that table.

   During encoding, the translation table's translations are applied to
the characters in the buffer, and the result of translation is actually
encoded.  If a coding system has property `:encode-translation-table',
that specifies the translation table to use, or a list of translation
tables to apply in sequence.  In addition, if the variable
`standard-translation-table-for-encode' is non-`nil', it specifies the
translation table to use for translating the result.

 -- Variable: standard-translation-table-for-decode
     This is the default translation table for decoding.  If a coding
     systems specifies its own translation tables, the table that is the
     value of this variable, if non-`nil', is applied after them.

 -- Variable: standard-translation-table-for-encode
     This is the default translation table for encoding.  If a coding
     systems specifies its own translation tables, the table that is the
     value of this variable, if non-`nil', is applied after them.

 -- Variable: translation-table-for-input
     Self-inserting characters are translated through this translation
     table before they are inserted.  Search commands also translate
     their input through this table, so they can compare more reliably
     with what's in the buffer.

     This variable automatically becomes buffer-local when set.

 -- Function: make-translation-table-from-vector vec
     This function returns a translation table made from VEC that is an
     array of 256 elements to map bytes (values 0 through #xFF) to
     characters.  Elements may be `nil' for untranslated bytes.  The
     returned table has a translation table for reverse mapping in the
     first extra slot, and the value `1' in the second extra slot.

     This function provides an easy way to make a private coding system
     that maps each byte to a specific character.  You can specify the
     returned table and the reverse translation table using the
     properties `:decode-translation-table' and
     `:encode-translation-table' respectively in the PROPS argument to
     `define-coding-system'.

 -- Function: make-translation-table-from-alist alist
     This function is similar to `make-translation-table' but returns a
     complex translation table rather than a simple one-to-one mapping.
     Each element of ALIST is of the form `(FROM . TO)', where FROM and
     TO are either characters or vectors specifying a sequence of
     characters.  If FROM is a character, that character is translated
     to TO (i.e. to a character or a character sequence).  If FROM is a
     vector of characters, that sequence is translated to TO.  The
     returned table has a translation table for reverse mapping in the
     first extra slot, and the maximum length of all the FROM character
     sequences in the second extra slot.


File: elisp,  Node: Coding Systems,  Next: Input Methods,  Prev: Translation of Characters,  Up: Non-ASCII Characters

33.9 Coding Systems
===================

When Emacs reads or writes a file, and when Emacs sends text to a
subprocess or receives text from a subprocess, it normally performs
character code conversion and end-of-line conversion as specified by a
particular "coding system".

   How to define a coding system is an arcane matter, and is not
documented here.

* Menu:

* Coding System Basics::        Basic concepts.
* Encoding and I/O::            How file I/O functions handle coding systems.
* Lisp and Coding Systems::     Functions to operate on coding system names.
* User-Chosen Coding Systems::  Asking the user to choose a coding system.
* Default Coding Systems::      Controlling the default choices.
* Specifying Coding Systems::   Requesting a particular coding system
                                    for a single file operation.
* Explicit Encoding::           Encoding or decoding text without doing I/O.
* Terminal I/O Encoding::       Use of encoding for terminal I/O.
* MS-DOS File Types::           How DOS "text" and "binary" files
                                    relate to coding systems.


File: elisp,  Node: Coding System Basics,  Next: Encoding and I/O,  Up: Coding Systems

33.9.1 Basic Concepts of Coding Systems
---------------------------------------

"Character code conversion" involves conversion between the internal
representation of characters used inside Emacs and some other encoding.
Emacs supports many different encodings, in that it can convert to and
from them.  For example, it can convert text to or from encodings such
as Latin 1, Latin 2, Latin 3, Latin 4, Latin 5, and several variants of
ISO 2022.  In some cases, Emacs supports several alternative encodings
for the same characters; for example, there are three coding systems
for the Cyrillic (Russian) alphabet: ISO, Alternativnyj, and KOI8.

   Every coding system specifies a particular set of character code
conversions, but the coding system `undecided' is special: it leaves
the choice unspecified, to be chosen heuristically for each file, based
on the file's data.

   In general, a coding system doesn't guarantee roundtrip identity:
decoding a byte sequence using coding system, then encoding the
resulting text in the same coding system, can produce a different byte
sequence.  But some coding systems do guarantee that the byte sequence
will be the same as what you originally decoded.  Here are a few
examples:

     iso-8859-1, utf-8, big5, shift_jis, euc-jp

   Encoding buffer text and then decoding the result can also fail to
reproduce the original text.  For instance, if you encode a character
with a coding system which does not support that character, the result
is unpredictable, and thus decoding it using the same coding system may
produce a different text.  Currently, Emacs can't report errors that
result from encoding unsupported characters.

   "End of line conversion" handles three different conventions used on
various systems for representing end of line in files.  The Unix
convention, used on GNU and Unix systems, is to use the linefeed
character (also called newline).  The DOS convention, used on
MS-Windows and MS-DOS systems, is to use a carriage-return and a
linefeed at the end of a line.  The Mac convention is to use just
carriage-return.

   "Base coding systems" such as `latin-1' leave the end-of-line
conversion unspecified, to be chosen based on the data.  "Variant
coding systems" such as `latin-1-unix', `latin-1-dos' and `latin-1-mac'
specify the end-of-line conversion explicitly as well.  Most base
coding systems have three corresponding variants whose names are formed
by adding `-unix', `-dos' and `-mac'.

   The coding system `raw-text' is special in that it prevents
character code conversion, and causes the buffer visited with this
coding system to be a unibyte buffer.  For historical reasons, you can
save both unibyte and multibyte text with this coding system.  When you
use `raw-text' to encode multibyte text, it does perform one character
code conversion: it converts eight-bit characters to their single-byte
external representation.  `raw-text' does not specify the end-of-line
conversion, allowing that to be determined as usual by the data, and
has the usual three variants which specify the end-of-line conversion.

   `no-conversion' (and its alias `binary') is equivalent to
`raw-text-unix': it specifies no conversion of either character codes
or end-of-line.

   The coding system `utf-8-emacs' specifies that the data is
represented in the internal Emacs encoding (*note Text
Representations::).  This is like `raw-text' in that no code conversion
happens, but different in that the result is multibyte data.  The name
`emacs-internal' is an alias for `utf-8-emacs'.

 -- Function: coding-system-get coding-system property
     This function returns the specified property of the coding system
     CODING-SYSTEM.  Most coding system properties exist for internal
     purposes, but one that you might find useful is `:mime-charset'.
     That property's value is the name used in MIME for the character
     coding which this coding system can read and write.  Examples:

          (coding-system-get 'iso-latin-1 :mime-charset)
               => iso-8859-1
          (coding-system-get 'iso-2022-cn :mime-charset)
               => iso-2022-cn
          (coding-system-get 'cyrillic-koi8 :mime-charset)
               => koi8-r

     The value of the `:mime-charset' property is also defined as an
     alias for the coding system.

 -- Function: coding-system-aliases coding-system
     This function returns the list of aliases of CODING-SYSTEM.


File: elisp,  Node: Encoding and I/O,  Next: Lisp and Coding Systems,  Prev: Coding System Basics,  Up: Coding Systems

33.9.2 Encoding and I/O
-----------------------

The principal purpose of coding systems is for use in reading and
writing files.  The function `insert-file-contents' uses a coding
system to decode the file data, and `write-region' uses one to encode
the buffer contents.

   You can specify the coding system to use either explicitly (*note
Specifying Coding Systems::), or implicitly using a default mechanism
(*note Default Coding Systems::).  But these methods may not completely
specify what to do.  For example, they may choose a coding system such
as `undefined' which leaves the character code conversion to be
determined from the data.  In these cases, the I/O operation finishes
the job of choosing a coding system.  Very often you will want to find
out afterwards which coding system was chosen.

 -- Variable: buffer-file-coding-system
     This buffer-local variable records the coding system used for
     saving the buffer and for writing part of the buffer with
     `write-region'.  If the text to be written cannot be safely
     encoded using the coding system specified by this variable, these
     operations select an alternative encoding by calling the function
     `select-safe-coding-system' (*note User-Chosen Coding Systems::).
     If selecting a different encoding requires to ask the user to
     specify a coding system, `buffer-file-coding-system' is updated to
     the newly selected coding system.

     `buffer-file-coding-system' does _not_ affect sending text to a
     subprocess.

 -- Variable: save-buffer-coding-system
     This variable specifies the coding system for saving the buffer (by
     overriding `buffer-file-coding-system').  Note that it is not used
     for `write-region'.

     When a command to save the buffer starts out to use
     `buffer-file-coding-system' (or `save-buffer-coding-system'), and
     that coding system cannot handle the actual text in the buffer,
     the command asks the user to choose another coding system (by
     calling `select-safe-coding-system').  After that happens, the
     command also updates `buffer-file-coding-system' to represent the
     coding system that the user specified.

 -- Variable: last-coding-system-used
     I/O operations for files and subprocesses set this variable to the
     coding system name that was used.  The explicit encoding and
     decoding functions (*note Explicit Encoding::) set it too.

     *Warning:* Since receiving subprocess output sets this variable,
     it can change whenever Emacs waits; therefore, you should copy the
     value shortly after the function call that stores the value you are
     interested in.

   The variable `selection-coding-system' specifies how to encode
selections for the window system.  *Note Window System Selections::.

 -- Variable: file-name-coding-system
     The variable `file-name-coding-system' specifies the coding system
     to use for encoding file names.  Emacs encodes file names using
     that coding system for all file operations.  If
     `file-name-coding-system' is `nil', Emacs uses a default coding
     system determined by the selected language environment.  In the
     default language environment, any non-ASCII characters in file
     names are not encoded specially; they appear in the file system
     using the internal Emacs representation.

   *Warning:* if you change `file-name-coding-system' (or the language
environment) in the middle of an Emacs session, problems can result if
you have already visited files whose names were encoded using the
earlier coding system and are handled differently under the new coding
system.  If you try to save one of these buffers under the visited file
name, saving may use the wrong file name, or it may get an error.  If
such a problem happens, use `C-x C-w' to specify a new file name for
that buffer.


File: elisp,  Node: Lisp and Coding Systems,  Next: User-Chosen Coding Systems,  Prev: Encoding and I/O,  Up: Coding Systems

33.9.3 Coding Systems in Lisp
-----------------------------

Here are the Lisp facilities for working with coding systems:

 -- Function: coding-system-list &optional base-only
     This function returns a list of all coding system names (symbols).
     If BASE-ONLY is non-`nil', the value includes only the base coding
     systems.  Otherwise, it includes alias and variant coding systems
     as well.

 -- Function: coding-system-p object
     This function returns `t' if OBJECT is a coding system name or
     `nil'.

 -- Function: check-coding-system coding-system
     This function checks the validity of CODING-SYSTEM.  If that is
     valid, it returns CODING-SYSTEM.  If CODING-SYSTEM is `nil', the
     function return `nil'.  For any other values, it signals an error
     whose `error-symbol' is `coding-system-error' (*note signal:
     Signaling Errors.).

 -- Function: coding-system-eol-type coding-system
     This function returns the type of end-of-line (a.k.a. "eol")
     conversion used by CODING-SYSTEM.  If CODING-SYSTEM specifies a
     certain eol conversion, the return value is an integer 0, 1, or 2,
     standing for `unix', `dos', and `mac', respectively.  If
     CODING-SYSTEM doesn't specify eol conversion explicitly, the
     return value is a vector of coding systems, each one with one of
     the possible eol conversion types, like this:

          (coding-system-eol-type 'latin-1)
               => [latin-1-unix latin-1-dos latin-1-mac]

     If this function returns a vector, Emacs will decide, as part of
     the text encoding or decoding process, what eol conversion to use.
     For decoding, the end-of-line format of the text is auto-detected,
     and the eol conversion is set to match it (e.g., DOS-style CRLF
     format will imply `dos' eol conversion).  For encoding, the eol
     conversion is taken from the appropriate default coding system
     (e.g., default value of `buffer-file-coding-system' for
     `buffer-file-coding-system'), or from the default eol conversion
     appropriate for the underlying platform.

 -- Function: coding-system-change-eol-conversion coding-system eol-type
     This function returns a coding system which is like CODING-SYSTEM
     except for its eol conversion, which is specified by `eol-type'.
     EOL-TYPE should be `unix', `dos', `mac', or `nil'.  If it is
     `nil', the returned coding system determines the end-of-line
     conversion from the data.

     EOL-TYPE may also be 0, 1 or 2, standing for `unix', `dos' and
     `mac', respectively.

 -- Function: coding-system-change-text-conversion eol-coding
          text-coding
     This function returns a coding system which uses the end-of-line
     conversion of EOL-CODING, and the text conversion of TEXT-CODING.
     If TEXT-CODING is `nil', it returns `undecided', or one of its
     variants according to EOL-CODING.

 -- Function: find-coding-systems-region from to
     This function returns a list of coding systems that could be used
     to encode a text between FROM and TO.  All coding systems in the
     list can safely encode any multibyte characters in that portion of
     the text.

     If the text contains no multibyte characters, the function returns
     the list `(undecided)'.

 -- Function: find-coding-systems-string string
     This function returns a list of coding systems that could be used
     to encode the text of STRING.  All coding systems in the list can
     safely encode any multibyte characters in STRING.  If the text
     contains no multibyte characters, this returns the list
     `(undecided)'.

 -- Function: find-coding-systems-for-charsets charsets
     This function returns a list of coding systems that could be used
     to encode all the character sets in the list CHARSETS.

 -- Function: check-coding-systems-region start end coding-system-list
     This function checks whether coding systems in the list
     `coding-system-list' can encode all the characters in the region
     between START and END.  If all of the coding systems in the list
     can encode the specified text, the function returns `nil'.  If
     some coding systems cannot encode some of the characters, the
     value is an alist, each element of which has the form
     `(CODING-SYSTEM1 POS1 POS2 ...)', meaning that CODING-SYSTEM1
     cannot encode characters at buffer positions POS1, POS2, ....

     START may be a string, in which case END is ignored and the
     returned value references string indices instead of buffer
     positions.

 -- Function: detect-coding-region start end &optional highest
     This function chooses a plausible coding system for decoding the
     text from START to END.  This text should be a byte sequence, i.e.
     unibyte text or multibyte text with only ASCII and eight-bit
     characters (*note Explicit Encoding::).

     Normally this function returns a list of coding systems that could
     handle decoding the text that was scanned.  They are listed in
     order of decreasing priority.  But if HIGHEST is non-`nil', then
     the return value is just one coding system, the one that is
     highest in priority.

     If the region contains only ASCII characters except for such
     ISO-2022 control characters ISO-2022 as `ESC', the value is
     `undecided' or `(undecided)', or a variant specifying end-of-line
     conversion, if that can be deduced from the text.

     If the region contains null bytes, the value is `no-conversion',
     even if the region contains text encoded in some coding system.

 -- Function: detect-coding-string string &optional highest
     This function is like `detect-coding-region' except that it
     operates on the contents of STRING instead of bytes in the buffer.

 -- Variable: inhibit-null-byte-detection
     If this variable has a non-`nil' value, null bytes are ignored
     when detecting the encoding of a region or a string.  This allows
     to correctly detect the encoding of text that contains null bytes,
     such as Info files with Index nodes.

 -- Variable: inhibit-iso-escape-detection
     If this variable has a non-`nil' value, ISO-2022 escape sequences
     are ignored when detecting the encoding of a region or a string.
     The result is that no text is ever detected as encoded in some
     ISO-2022 encoding, and all escape sequences become visible in a
     buffer.  *Warning:* _Use this variable with extreme caution,
     because many files in the Emacs distribution use ISO-2022
     encoding._

 -- Function: coding-system-charset-list coding-system
     This function returns the list of character sets (*note Character
     Sets::) supported by CODING-SYSTEM.  Some coding systems that
     support too many character sets to list them all yield special
     values:
        * If CODING-SYSTEM supports all the ISO-2022 charsets, the value
          is `iso-2022'.

        * If CODING-SYSTEM supports all Emacs characters, the value is
          `(emacs)'.

        * If CODING-SYSTEM supports all emacs-mule characters, the value
          is `emacs-mule'.

        * If CODING-SYSTEM supports all Unicode characters, the value is
          `(unicode)'.

   *Note Process Information: Coding systems for a subprocess, in
particular the description of the functions `process-coding-system' and
`set-process-coding-system', for how to examine or set the coding
systems used for I/O to a subprocess.


File: elisp,  Node: User-Chosen Coding Systems,  Next: Default Coding Systems,  Prev: Lisp and Coding Systems,  Up: Coding Systems

33.9.4 User-Chosen Coding Systems
---------------------------------

 -- Function: select-safe-coding-system from to &optional
          default-coding-system accept-default-p file
     This function selects a coding system for encoding specified text,
     asking the user to choose if necessary.  Normally the specified
     text is the text in the current buffer between FROM and TO.  If
     FROM is a string, the string specifies the text to encode, and TO
     is ignored.

     If the specified text includes raw bytes (*note Text
     Representations::), `select-safe-coding-system' suggests
     `raw-text' for its encoding.

     If DEFAULT-CODING-SYSTEM is non-`nil', that is the first coding
     system to try; if that can handle the text,
     `select-safe-coding-system' returns that coding system.  It can
     also be a list of coding systems; then the function tries each of
     them one by one.  After trying all of them, it next tries the
     current buffer's value of `buffer-file-coding-system' (if it is not
     `undecided'), then the default value of
     `buffer-file-coding-system' and finally the user's most preferred
     coding system, which the user can set using the command
     `prefer-coding-system' (*note Recognizing Coding Systems:
     (emacs)Recognize Coding.).

     If one of those coding systems can safely encode all the specified
     text, `select-safe-coding-system' chooses it and returns it.
     Otherwise, it asks the user to choose from a list of coding systems
     which can encode all the text, and returns the user's choice.

     DEFAULT-CODING-SYSTEM can also be a list whose first element is t
     and whose other elements are coding systems.  Then, if no coding
     system in the list can handle the text, `select-safe-coding-system'
     queries the user immediately, without trying any of the three
     alternatives described above.

     The optional argument ACCEPT-DEFAULT-P, if non-`nil', should be a
     function to determine whether a coding system selected without
     user interaction is acceptable. `select-safe-coding-system' calls
     this function with one argument, the base coding system of the
     selected coding system.  If ACCEPT-DEFAULT-P returns `nil',
     `select-safe-coding-system' rejects the silently selected coding
     system, and asks the user to select a coding system from a list of
     possible candidates.

     If the variable `select-safe-coding-system-accept-default-p' is
     non-`nil', it should be a function taking a single argument.  It
     is used in place of ACCEPT-DEFAULT-P, overriding any value
     supplied for this argument.

     As a final step, before returning the chosen coding system,
     `select-safe-coding-system' checks whether that coding system is
     consistent with what would be selected if the contents of the
     region were read from a file.  (If not, this could lead to data
     corruption in a file subsequently re-visited and edited.)
     Normally, `select-safe-coding-system' uses `buffer-file-name' as
     the file for this purpose, but if FILE is non-`nil', it uses that
     file instead (this can be relevant for `write-region' and similar
     functions).  If it detects an apparent inconsistency,
     `select-safe-coding-system' queries the user before selecting the
     coding system.

   Here are two functions you can use to let the user specify a coding
system, with completion.  *Note Completion::.

 -- Function: read-coding-system prompt &optional default
     This function reads a coding system using the minibuffer,
     prompting with string PROMPT, and returns the coding system name
     as a symbol.  If the user enters null input, DEFAULT specifies
     which coding system to return.  It should be a symbol or a string.

 -- Function: read-non-nil-coding-system prompt
     This function reads a coding system using the minibuffer,
     prompting with string PROMPT, and returns the coding system name
     as a symbol.  If the user tries to enter null input, it asks the
     user to try again.  *Note Coding Systems::.


File: elisp,  Node: Default Coding Systems,  Next: Specifying Coding Systems,  Prev: User-Chosen Coding Systems,  Up: Coding Systems

33.9.5 Default Coding Systems
-----------------------------

This section describes variables that specify the default coding system
for certain files or when running certain subprograms, and the function
that I/O operations use to access them.

   The idea of these variables is that you set them once and for all to
the defaults you want, and then do not change them again.  To specify a
particular coding system for a particular operation in a Lisp program,
don't change these variables; instead, override them using
`coding-system-for-read' and `coding-system-for-write' (*note
Specifying Coding Systems::).

 -- User Option: auto-coding-regexp-alist
     This variable is an alist of text patterns and corresponding coding
     systems. Each element has the form `(REGEXP . CODING-SYSTEM)'; a
     file whose first few kilobytes match REGEXP is decoded with
     CODING-SYSTEM when its contents are read into a buffer.  The
     settings in this alist take priority over `coding:' tags in the
     files and the contents of `file-coding-system-alist' (see below).
     The default value is set so that Emacs automatically recognizes
     mail files in Babyl format and reads them with no code conversions.

 -- User Option: file-coding-system-alist
     This variable is an alist that specifies the coding systems to use
     for reading and writing particular files.  Each element has the
     form `(PATTERN . CODING)', where PATTERN is a regular expression
     that matches certain file names.  The element applies to file
     names that match PATTERN.

     The CDR of the element, CODING, should be either a coding system,
     a cons cell containing two coding systems, or a function name (a
     symbol with a function definition).  If CODING is a coding system,
     that coding system is used for both reading the file and writing
     it.  If CODING is a cons cell containing two coding systems, its
     CAR specifies the coding system for decoding, and its CDR
     specifies the coding system for encoding.

     If CODING is a function name, the function should take one
     argument, a list of all arguments passed to
     `find-operation-coding-system'.  It must return a coding system or
     a cons cell containing two coding systems.  This value has the same
     meaning as described above.

     If CODING (or what returned by the above function) is `undecided',
     the normal code-detection is performed.

 -- User Option: auto-coding-alist
     This variable is an alist that specifies the coding systems to use
     for reading and writing particular files.  Its form is like that of
     `file-coding-system-alist', but, unlike the latter, this variable
     takes priority over any `coding:' tags in the file.

 -- Variable: process-coding-system-alist
     This variable is an alist specifying which coding systems to use
     for a subprocess, depending on which program is running in the
     subprocess.  It works like `file-coding-system-alist', except that
     PATTERN is matched against the program name used to start the
     subprocess.  The coding system or systems specified in this alist
     are used to initialize the coding systems used for I/O to the
     subprocess, but you can specify other coding systems later using
     `set-process-coding-system'.

   *Warning:* Coding systems such as `undecided', which determine the
coding system from the data, do not work entirely reliably with
asynchronous subprocess output.  This is because Emacs handles
asynchronous subprocess output in batches, as it arrives.  If the coding
system leaves the character code conversion unspecified, or leaves the
end-of-line conversion unspecified, Emacs must try to detect the proper
conversion from one batch at a time, and this does not always work.

   Therefore, with an asynchronous subprocess, if at all possible, use a
coding system which determines both the character code conversion and
the end of line conversion--that is, one like `latin-1-unix', rather
than `undecided' or `latin-1'.

 -- Variable: network-coding-system-alist
     This variable is an alist that specifies the coding system to use
     for network streams.  It works much like
     `file-coding-system-alist', with the difference that the PATTERN
     in an element may be either a port number or a regular expression.
     If it is a regular expression, it is matched against the network
     service name used to open the network stream.

 -- Variable: default-process-coding-system
     This variable specifies the coding systems to use for subprocess
     (and network stream) input and output, when nothing else specifies
     what to do.

     The value should be a cons cell of the form `(INPUT-CODING .
     OUTPUT-CODING)'.  Here INPUT-CODING applies to input from the
     subprocess, and OUTPUT-CODING applies to output to it.

 -- User Option: auto-coding-functions
     This variable holds a list of functions that try to determine a
     coding system for a file based on its undecoded contents.

     Each function in this list should be written to look at text in the
     current buffer, but should not modify it in any way.  The buffer
     will contain undecoded text of parts of the file.  Each function
     should take one argument, SIZE, which tells it how many characters
     to look at, starting from point.  If the function succeeds in
     determining a coding system for the file, it should return that
     coding system.  Otherwise, it should return `nil'.

     If a file has a `coding:' tag, that takes precedence, so these
     functions won't be called.

 -- Function: find-auto-coding filename size
     This function tries to determine a suitable coding system for
     FILENAME.  It examines the buffer visiting the named file, using
     the variables documented above in sequence, until it finds a match
     for one of the rules specified by these variables.  It then
     returns a cons cell of the form `(CODING . SOURCE)', where CODING
     is the coding system to use and SOURCE is a symbol, one of
     `auto-coding-alist', `auto-coding-regexp-alist', `:coding', or
     `auto-coding-functions', indicating which one supplied the
     matching rule.  The value `:coding' means the coding system was
     specified by the `coding:' tag in the file (*note coding tag:
     (emacs)Specify Coding.).  The order of looking for a matching rule
     is `auto-coding-alist' first, then `auto-coding-regexp-alist',
     then the `coding:' tag, and lastly `auto-coding-functions'.  If no
     matching rule was found, the function returns `nil'.

     The second argument SIZE is the size of text, in characters,
     following point.  The function examines text only within SIZE
     characters after point.  Normally, the buffer should be positioned
     at the beginning when this function is called, because one of the
     places for the `coding:' tag is the first one or two lines of the
     file; in that case, SIZE should be the size of the buffer.

 -- Function: set-auto-coding filename size
     This function returns a suitable coding system for file FILENAME.
     It uses `find-auto-coding' to find the coding system.  If no
     coding system could be determined, the function returns `nil'.
     The meaning of the argument SIZE is like in `find-auto-coding'.

 -- Function: find-operation-coding-system operation &rest arguments
     This function returns the coding system to use (by default) for
     performing OPERATION with ARGUMENTS.  The value has this form:

          (DECODING-SYSTEM . ENCODING-SYSTEM)

     The first element, DECODING-SYSTEM, is the coding system to use
     for decoding (in case OPERATION does decoding), and
     ENCODING-SYSTEM is the coding system for encoding (in case
     OPERATION does encoding).

     The argument OPERATION is a symbol; it should be one of
     `write-region', `start-process', `call-process',
     `call-process-region', `insert-file-contents', or
     `open-network-stream'.  These are the names of the Emacs I/O
     primitives that can do character code and eol conversion.

     The remaining arguments should be the same arguments that might be
     given to the corresponding I/O primitive.  Depending on the
     primitive, one of those arguments is selected as the "target".
     For example, if OPERATION does file I/O, whichever argument
     specifies the file name is the target.  For subprocess primitives,
     the process name is the target.  For `open-network-stream', the
     target is the service name or port number.

     Depending on OPERATION, this function looks up the target in
     `file-coding-system-alist', `process-coding-system-alist', or
     `network-coding-system-alist'.  If the target is found in the
     alist, `find-operation-coding-system' returns its association in
     the alist; otherwise it returns `nil'.

     If OPERATION is `insert-file-contents', the argument corresponding
     to the target may be a cons cell of the form `(FILENAME .
     BUFFER)').  In that case, FILENAME is a file name to look up in
     `file-coding-system-alist', and BUFFER is a buffer that contains
     the file's contents (not yet decoded).  If
     `file-coding-system-alist' specifies a function to call for this
     file, and that function needs to examine the file's contents (as
     it usually does), it should examine the contents of BUFFER instead
     of reading the file.


File: elisp,  Node: Specifying Coding Systems,  Next: Explicit Encoding,  Prev: Default Coding Systems,  Up: Coding Systems

33.9.6 Specifying a Coding System for One Operation
---------------------------------------------------

You can specify the coding system for a specific operation by binding
the variables `coding-system-for-read' and/or `coding-system-for-write'.

 -- Variable: coding-system-for-read
     If this variable is non-`nil', it specifies the coding system to
     use for reading a file, or for input from a synchronous subprocess.

     It also applies to any asynchronous subprocess or network stream,
     but in a different way: the value of `coding-system-for-read' when
     you start the subprocess or open the network stream specifies the
     input decoding method for that subprocess or network stream.  It
     remains in use for that subprocess or network stream unless and
     until overridden.

     The right way to use this variable is to bind it with `let' for a
     specific I/O operation.  Its global value is normally `nil', and
     you should not globally set it to any other value.  Here is an
     example of the right way to use the variable:

          ;; Read the file with no character code conversion.
          ;; Assume crlf represents end-of-line.
          (let ((coding-system-for-read 'emacs-mule-dos))
            (insert-file-contents filename))

     When its value is non-`nil', this variable takes precedence over
     all other methods of specifying a coding system to use for input,
     including `file-coding-system-alist',
     `process-coding-system-alist' and `network-coding-system-alist'.

 -- Variable: coding-system-for-write
     This works much like `coding-system-for-read', except that it
     applies to output rather than input.  It affects writing to files,
     as well as sending output to subprocesses and net connections.

     When a single operation does both input and output, as do
     `call-process-region' and `start-process', both
     `coding-system-for-read' and `coding-system-for-write' affect it.

 -- User Option: inhibit-eol-conversion
     When this variable is non-`nil', no end-of-line conversion is done,
     no matter which coding system is specified.  This applies to all
     the Emacs I/O and subprocess primitives, and to the explicit
     encoding and decoding functions (*note Explicit Encoding::).

   Sometimes, you need to prefer several coding systems for some
operation, rather than fix a single one.  Emacs lets you specify a
priority order for using coding systems.  This ordering affects the
sorting of lists of coding systems returned by functions such as
`find-coding-systems-region' (*note Lisp and Coding Systems::).

 -- Function: coding-system-priority-list &optional highestp
     This function returns the list of coding systems in the order of
     their current priorities.  Optional argument HIGHESTP, if
     non-`nil', means return only the highest priority coding system.

 -- Function: set-coding-system-priority &rest coding-systems
     This function puts CODING-SYSTEMS at the beginning of the priority
     list for coding systems, thus making their priority higher than
     all the rest.

 -- Macro: with-coding-priority coding-systems &rest body...
     This macro execute BODY, like `progn' does (*note progn:
     Sequencing.), with CODING-SYSTEMS at the front of the priority
     list for coding systems.  CODING-SYSTEMS should be a list of
     coding systems to prefer during execution of BODY.


File: elisp,  Node: Explicit Encoding,  Next: Terminal I/O Encoding,  Prev: Specifying Coding Systems,  Up: Coding Systems

33.9.7 Explicit Encoding and Decoding
-------------------------------------

All the operations that transfer text in and out of Emacs have the
ability to use a coding system to encode or decode the text.  You can
also explicitly encode and decode text using the functions in this
section.

   The result of encoding, and the input to decoding, are not ordinary
text.  They logically consist of a series of byte values; that is, a
series of ASCII and eight-bit characters.  In unibyte buffers and
strings, these characters have codes in the range 0 through #xFF (255).
In a multibyte buffer or string, eight-bit characters have character
codes higher than #xFF (*note Text Representations::), but Emacs
transparently converts them to their single-byte values when you encode
or decode such text.

   The usual way to read a file into a buffer as a sequence of bytes, so
you can decode the contents explicitly, is with
`insert-file-contents-literally' (*note Reading from Files::);
alternatively, specify a non-`nil' RAWFILE argument when visiting a
file with `find-file-noselect'.  These methods result in a unibyte
buffer.

   The usual way to use the byte sequence that results from explicitly
encoding text is to copy it to a file or process--for example, to write
it with `write-region' (*note Writing to Files::), and suppress
encoding by binding `coding-system-for-write' to `no-conversion'.

   Here are the functions to perform explicit encoding or decoding.  The
encoding functions produce sequences of bytes; the decoding functions
are meant to operate on sequences of bytes.  All of these functions
discard text properties.  They also set `last-coding-system-used' to
the precise coding system they used.

 -- Command: encode-coding-region start end coding-system &optional
          destination
     This command encodes the text from START to END according to
     coding system CODING-SYSTEM.  Normally, the encoded text replaces
     the original text in the buffer, but the optional argument
     DESTINATION can change that.  If DESTINATION is a buffer, the
     encoded text is inserted in that buffer after point (point does
     not move); if it is `t', the command returns the encoded text as a
     unibyte string without inserting it.

     If encoded text is inserted in some buffer, this command returns
     the length of the encoded text.

     The result of encoding is logically a sequence of bytes, but the
     buffer remains multibyte if it was multibyte before, and any 8-bit
     bytes are converted to their multibyte representation (*note Text
     Representations::).

     Do _not_ use `undecided' for CODING-SYSTEM when encoding text,
     since that may lead to unexpected results.  Instead, use
     `select-safe-coding-system' (*note select-safe-coding-system:
     User-Chosen Coding Systems.) to suggest a suitable encoding, if
     there's no obvious pertinent value for CODING-SYSTEM.

 -- Function: encode-coding-string string coding-system &optional
          nocopy buffer
     This function encodes the text in STRING according to coding
     system CODING-SYSTEM.  It returns a new string containing the
     encoded text, except when NOCOPY is non-`nil', in which case the
     function may return STRING itself if the encoding operation is
     trivial.  The result of encoding is a unibyte string.

 -- Command: decode-coding-region start end coding-system &optional
          destination
     This command decodes the text from START to END according to
     coding system CODING-SYSTEM.  To make explicit decoding useful,
     the text before decoding ought to be a sequence of byte values,
     but both multibyte and unibyte buffers are acceptable (in the
     multibyte case, the raw byte values should be represented as
     eight-bit characters).  Normally, the decoded text replaces the
     original text in the buffer, but the optional argument DESTINATION
     can change that.  If DESTINATION is a buffer, the decoded text is
     inserted in that buffer after point (point does not move); if it
     is `t', the command returns the decoded text as a multibyte string
     without inserting it.

     If decoded text is inserted in some buffer, this command returns
     the length of the decoded text.

     This command puts a `charset' text property on the decoded text.
     The value of the property states the character set used to decode
     the original text.

 -- Function: decode-coding-string string coding-system &optional
          nocopy buffer
     This function decodes the text in STRING according to
     CODING-SYSTEM.  It returns a new string containing the decoded
     text, except when NOCOPY is non-`nil', in which case the function
     may return STRING itself if the decoding operation is trivial.  To
     make explicit decoding useful, the contents of STRING ought to be
     a unibyte string with a sequence of byte values, but a multibyte
     string is also acceptable (assuming it contains 8-bit bytes in
     their multibyte form).

     If optional argument BUFFER specifies a buffer, the decoded text
     is inserted in that buffer after point (point does not move).  In
     this case, the return value is the length of the decoded text.

     This function puts a `charset' text property on the decoded text.
     The value of the property states the character set used to decode
     the original text:

          (decode-coding-string "Gr\374ss Gott" 'latin-1)
               => #("Gr�ss Gott" 0 9 (charset iso-8859-1))

 -- Function: decode-coding-inserted-region from to filename &optional
          visit beg end replace
     This function decodes the text from FROM to TO as if it were being
     read from file FILENAME using `insert-file-contents' using the
     rest of the arguments provided.

     The normal way to use this function is after reading text from a
     file without decoding, if you decide you would rather have decoded
     it.  Instead of deleting the text and reading it again, this time
     with decoding, you can call this function.


File: elisp,  Node: Terminal I/O Encoding,  Next: MS-DOS File Types,  Prev: Explicit Encoding,  Up: Coding Systems

33.9.8 Terminal I/O Encoding
----------------------------

Emacs can decode keyboard input using a coding system, and encode
terminal output.  This is useful for terminals that transmit or display
text using a particular encoding such as Latin-1.  Emacs does not set
`last-coding-system-used' for encoding or decoding of terminal I/O.

 -- Function: keyboard-coding-system &optional terminal
     This function returns the coding system that is in use for decoding
     keyboard input from TERMINAL--or `nil' if no coding system is to
     be used for that terminal.  If TERMINAL is omitted or `nil', it
     means the selected frame's terminal.  *Note Multiple Terminals::.

 -- Command: set-keyboard-coding-system coding-system &optional terminal
     This command specifies CODING-SYSTEM as the coding system to use
     for decoding keyboard input from TERMINAL.  If CODING-SYSTEM is
     `nil', that means do not decode keyboard input.  If TERMINAL is a
     frame, it means that frame's terminal; if it is `nil', that means
     the currently selected frame's terminal.  *Note Multiple
     Terminals::.

 -- Function: terminal-coding-system &optional terminal
     This function returns the coding system that is in use for encoding
     terminal output from TERMINAL--or `nil' if the output is not
     encoded.  If TERMINAL is a frame, it means that frame's terminal;
     if it is `nil', that means the currently selected frame's terminal.

 -- Command: set-terminal-coding-system coding-system &optional terminal
     This command specifies CODING-SYSTEM as the coding system to use
     for encoding terminal output from TERMINAL.  If CODING-SYSTEM is
     `nil', terminal output is not encoded.  If TERMINAL is a frame, it
     means that frame's terminal; if it is `nil', that means the
     currently selected frame's terminal.


File: elisp,  Node: MS-DOS File Types,  Prev: Terminal I/O Encoding,  Up: Coding Systems

33.9.9 MS-DOS File Types
------------------------

On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
end-of-line conversion for a file by looking at the file's name.  This
feature classifies files as "text files" and "binary files".  By
"binary file" we mean a file of literal byte values that are not
necessarily meant to be characters; Emacs does no end-of-line conversion
and no character code conversion for them.  On the other hand, the bytes
in a text file are intended to represent characters; when you create a
new file whose name implies that it is a text file, Emacs uses DOS
end-of-line conversion.

 -- Variable: buffer-file-type
     This variable, automatically buffer-local in each buffer, records
     the file type of the buffer's visited file.  When a buffer does
     not specify a coding system with `buffer-file-coding-system', this
     variable is used to determine which coding system to use when
     writing the contents of the buffer.  It should be `nil' for text,
     `t' for binary.  If it is `t', the coding system is
     `no-conversion'.  Otherwise, `undecided-dos' is used.

     Normally this variable is set by visiting a file; it is set to
     `nil' if the file was visited without any actual conversion.

     Its default value is used to decide how to handle files for which
     `file-name-buffer-file-type-alist' says nothing about the type: If
     the default value is non-`nil', then these files are treated as
     binary: the coding system `no-conversion' is used.  Otherwise,
     nothing special is done for them--the coding system is deduced
     solely from the file contents, in the usual Emacs fashion.

 -- User Option: file-name-buffer-file-type-alist
     This variable holds an alist for recognizing text and binary files.
     Each element has the form (REGEXP . TYPE), where REGEXP is matched
     against the file name, and TYPE may be `nil' for text, `t' for
     binary, or a function to call to compute which.  If it is a
     function, then it is called with a single argument (the file name)
     and should return `t' or `nil'.

     When running on MS-DOS or MS-Windows, Emacs checks this alist to
     decide which coding system to use when reading a file.  For a text
     file, `undecided-dos' is used.  For a binary file, `no-conversion'
     is used.

     If no element in this alist matches a given file name, then the
     default value of `buffer-file-type' says how to treat the file.


File: elisp,  Node: Input Methods,  Next: Locales,  Prev: Coding Systems,  Up: Non-ASCII Characters

33.10 Input Methods
===================

"Input methods" provide convenient ways of entering non-ASCII
characters from the keyboard.  Unlike coding systems, which translate
non-ASCII characters to and from encodings meant to be read by
programs, input methods provide human-friendly commands.  (*Note Input
Methods: (emacs)Input Methods, for information on how users use input
methods to enter text.)  How to define input methods is not yet
documented in this manual, but here we describe how to use them.

   Each input method has a name, which is currently a string; in the
future, symbols may also be usable as input method names.

 -- Variable: current-input-method
     This variable holds the name of the input method now active in the
     current buffer.  (It automatically becomes local in each buffer
     when set in any fashion.)  It is `nil' if no input method is
     active in the buffer now.

 -- User Option: default-input-method
     This variable holds the default input method for commands that
     choose an input method.  Unlike `current-input-method', this
     variable is normally global.

 -- Command: set-input-method input-method
     This command activates input method INPUT-METHOD for the current
     buffer.  It also sets `default-input-method' to INPUT-METHOD.  If
     INPUT-METHOD is `nil', this command deactivates any input method
     for the current buffer.

 -- Function: read-input-method-name prompt &optional default
          inhibit-null
     This function reads an input method name with the minibuffer,
     prompting with PROMPT.  If DEFAULT is non-`nil', that is returned
     by default, if the user enters empty input.  However, if
     INHIBIT-NULL is non-`nil', empty input signals an error.

     The returned value is a string.

 -- Variable: input-method-alist
     This variable defines all the supported input methods.  Each
     element defines one input method, and should have the form:

          (INPUT-METHOD LANGUAGE-ENV ACTIVATE-FUNC
           TITLE DESCRIPTION ARGS...)

     Here INPUT-METHOD is the input method name, a string; LANGUAGE-ENV
     is another string, the name of the language environment this input
     method is recommended for.  (That serves only for documentation
     purposes.)

     ACTIVATE-FUNC is a function to call to activate this method.  The
     ARGS, if any, are passed as arguments to ACTIVATE-FUNC.  All told,
     the arguments to ACTIVATE-FUNC are INPUT-METHOD and the ARGS.

     TITLE is a string to display in the mode line while this method is
     active.  DESCRIPTION is a string describing this method and what
     it is good for.

   The fundamental interface to input methods is through the variable
`input-method-function'.  *Note Reading One Event::, and *note Invoking
the Input Method::.


File: elisp,  Node: Locales,  Prev: Input Methods,  Up: Non-ASCII Characters

33.11 Locales
=============

POSIX defines a concept of "locales" which control which language to
use in language-related features.  These Emacs variables control how
Emacs interacts with these features.

 -- Variable: locale-coding-system
     This variable specifies the coding system to use for decoding
     system error messages and--on X Window system only--keyboard
     input, for encoding the format argument to `format-time-string',
     and for decoding the return value of `format-time-string'.

 -- Variable: system-messages-locale
     This variable specifies the locale to use for generating system
     error messages.  Changing the locale can cause messages to come
     out in a different language or in a different orthography.  If the
     variable is `nil', the locale is specified by environment
     variables in the usual POSIX fashion.

 -- Variable: system-time-locale
     This variable specifies the locale to use for formatting time
     values.  Changing the locale can cause messages to appear
     according to the conventions of a different language.  If the
     variable is `nil', the locale is specified by environment
     variables in the usual POSIX fashion.

 -- Function: locale-info item
     This function returns locale data ITEM for the current POSIX
     locale, if available.  ITEM should be one of these symbols:

    `codeset'
          Return the character set as a string (locale item `CODESET').

    `days'
          Return a 7-element vector of day names (locale items `DAY_1'
          through `DAY_7');

    `months'
          Return a 12-element vector of month names (locale items
          `MON_1' through `MON_12').

    `paper'
          Return a list `(WIDTH HEIGHT)' for the default paper size
          measured in millimeters (locale items `PAPER_WIDTH' and
          `PAPER_HEIGHT').

     If the system can't provide the requested information, or if ITEM
     is not one of those symbols, the value is `nil'.  All strings in
     the return value are decoded using `locale-coding-system'.  *Note
     Locales: (libc)Locales, for more information about locales and
     locale items.


File: elisp,  Node: Searching and Matching,  Next: Syntax Tables,  Prev: Non-ASCII Characters,  Up: Top

34 Searching and Matching
*************************

GNU Emacs provides two ways to search through a buffer for specified
text: exact string searches and regular expression searches.  After a
regular expression search, you can examine the "match data" to
determine which text matched the whole regular expression or various
portions of it.

* Menu:

* String Search::         Search for an exact match.
* Searching and Case::    Case-independent or case-significant searching.
* Regular Expressions::   Describing classes of strings.
* Regexp Search::         Searching for a match for a regexp.
* POSIX Regexps::         Searching POSIX-style for the longest match.
* Match Data::            Finding out which part of the text matched,
                            after a string or regexp search.
* Search and Replace::    Commands that loop, searching and replacing.
* Standard Regexps::      Useful regexps for finding sentences, pages,...

   The `skip-chars...' functions also perform a kind of searching.
*Note Skipping Characters::.  To search for changes in character
properties, see *note Property Search::.


File: elisp,  Node: String Search,  Next: Searching and Case,  Up: Searching and Matching

34.1 Searching for Strings
==========================

These are the primitive functions for searching through the text in a
buffer.  They are meant for use in programs, but you may call them
interactively.  If you do so, they prompt for the search string; the
arguments LIMIT and NOERROR are `nil', and REPEAT is 1.  For more
details on interactive searching, *note Searching and Replacement:
(emacs)Search.

   These search functions convert the search string to multibyte if the
buffer is multibyte; they convert the search string to unibyte if the
buffer is unibyte.  *Note Text Representations::.

 -- Command: search-forward string &optional limit noerror repeat
     This function searches forward from point for an exact match for
     STRING.  If successful, it sets point to the end of the occurrence
     found, and returns the new value of point.  If no match is found,
     the value and side effects depend on NOERROR (see below).

     In the following example, point is initially at the beginning of
     the line.  Then `(search-forward "fox")' moves point after the last
     letter of `fox':

          ---------- Buffer: foo ----------
          -!-The quick brown fox jumped over the lazy dog.
          ---------- Buffer: foo ----------

          (search-forward "fox")
               => 20

          ---------- Buffer: foo ----------
          The quick brown fox-!- jumped over the lazy dog.
          ---------- Buffer: foo ----------

     The argument LIMIT specifies the bound to the search, and should
     be a position in the current buffer.  No match extending after
     that position is accepted.  If LIMIT is omitted or `nil', it
     defaults to the end of the accessible portion of the buffer.

     What happens when the search fails depends on the value of
     NOERROR.  If NOERROR is `nil', a `search-failed' error is
     signaled.  If NOERROR is `t', `search-forward' returns `nil' and
     does nothing.  If NOERROR is neither `nil' nor `t', then
     `search-forward' moves point to the upper bound and returns `nil'.

     The argument NOERROR only affects valid searches which fail to
     find a match.  Invalid arguments cause errors regardless of
     NOERROR.

     If REPEAT is a positive number N, it serves as a repeat count: the
     search is repeated N times, each time starting at the end of the
     previous time's match.  If these successive searches succeed, the
     function succeeds, moving point and returning its new value.
     Otherwise the search fails, with results depending on the value of
     NOERROR, as described above.  If REPEAT is a negative number -N,
     it serves as a repeat count of N for a search in the opposite
     (backward) direction.

 -- Command: search-backward string &optional limit noerror repeat
     This function searches backward from point for STRING.  It is like
     `search-forward', except that it searches backwards rather than
     forwards.  Backward searches leave point at the beginning of the
     match.

 -- Command: word-search-forward string &optional limit noerror repeat
     This function searches forward from point for a "word" match for
     STRING.  If it finds a match, it sets point to the end of the
     match found, and returns the new value of point.

     Word matching regards STRING as a sequence of words, disregarding
     punctuation that separates them.  It searches the buffer for the
     same sequence of words.  Each word must be distinct in the buffer
     (searching for the word `ball' does not match the word `balls'),
     but the details of punctuation and spacing are ignored (searching
     for `ball boy' does match `ball.  Boy!').

     In this example, point is initially at the beginning of the
     buffer; the search leaves it between the `y' and the `!'.

          ---------- Buffer: foo ----------
          -!-He said "Please!  Find
          the ball boy!"
          ---------- Buffer: foo ----------

          (word-search-forward "Please find the ball, boy.")
               => 36

          ---------- Buffer: foo ----------
          He said "Please!  Find
          the ball boy-!-!"
          ---------- Buffer: foo ----------

     If LIMIT is non-`nil', it must be a position in the current
     buffer; it specifies the upper bound to the search.  The match
     found must not extend after that position.

     If NOERROR is `nil', then `word-search-forward' signals an error
     if the search fails.  If NOERROR is `t', then it returns `nil'
     instead of signaling an error.  If NOERROR is neither `nil' nor
     `t', it moves point to LIMIT (or the end of the accessible portion
     of the buffer) and returns `nil'.

     If REPEAT is non-`nil', then the search is repeated that many
     times.  Point is positioned at the end of the last match.

     Internal, `word-search-forward' and related functions use the
     function `word-search-regexp' to convert STRING to a regular
     expression that ignores punctuation.

 -- Command: word-search-forward-lax string &optional limit noerror
          repeat
     This command is identical to `word-search-forward', except that
     the end of STRING need not match a word boundary, unless STRING
     ends in whitespace.  For instance, searching for `ball boy' matches
     `ball boyee', but does not match `aball boy'.

 -- Command: word-search-backward string &optional limit noerror repeat
     This function searches backward from point for a word match to
     STRING.  This function is just like `word-search-forward' except
     that it searches backward and normally leaves point at the
     beginning of the match.

 -- Command: word-search-backward-lax string &optional limit noerror
          repeat
     This command is identical to `word-search-backward', except that
     the end of STRING need not match a word boundary, unless STRING
     ends in whitespace.


File: elisp,  Node: Searching and Case,  Next: Regular Expressions,  Prev: String Search,  Up: Searching and Matching

34.2 Searching and Case
=======================

By default, searches in Emacs ignore the case of the text they are
searching through; if you specify searching for `FOO', then `Foo' or
`foo' is also considered a match.  This applies to regular expressions,
too; thus, `[aB]' would match `a' or `A' or `b' or `B'.

   If you do not want this feature, set the variable `case-fold-search'
to `nil'.  Then all letters must match exactly, including case.  This
is a buffer-local variable; altering the variable affects only the
current buffer.  (*Note Intro to Buffer-Local::.)  Alternatively, you
may change the default value.  In Lisp code, you will more typically
use `let' to bind `case-fold-search' to the desired value.

   Note that the user-level incremental search feature handles case
distinctions differently.  When the search string contains only lower
case letters, the search ignores case, but when the search string
contains one or more upper case letters, the search becomes
case-sensitive.  But this has nothing to do with the searching
functions used in Lisp code.  *Note Incremental Search:
(emacs)Incremental Search.

 -- User Option: case-fold-search
     This buffer-local variable determines whether searches should
     ignore case.  If the variable is `nil' they do not ignore case;
     otherwise (and by default) they do ignore case.

 -- User Option: case-replace
     This variable determines whether the higher-level replacement
     functions should preserve case.  If the variable is `nil', that
     means to use the replacement text verbatim.  A non-`nil' value
     means to convert the case of the replacement text according to the
     text being replaced.

     This variable is used by passing it as an argument to the function
     `replace-match'.  *Note Replacing Match::.


File: elisp,  Node: Regular Expressions,  Next: Regexp Search,  Prev: Searching and Case,  Up: Searching and Matching

34.3 Regular Expressions
========================

A "regular expression", or "regexp" for short, is a pattern that
denotes a (possibly infinite) set of strings.  Searching for matches for
a regexp is a very powerful operation.  This section explains how to
write regexps; the following section says how to search for them.

   For interactive development of regular expressions, you can use the
`M-x re-builder' command.  It provides a convenient interface for
creating regular expressions, by giving immediate visual feedback in a
separate buffer.  As you edit the regexp, all its matches in the target
buffer are highlighted.  Each parenthesized sub-expression of the
regexp is shown in a distinct face, which makes it easier to verify
even very complex regexps.

* Menu:

* Syntax of Regexps::       Rules for writing regular expressions.
* Regexp Example::          Illustrates regular expression syntax.
* Regexp Functions::        Functions for operating on regular expressions.


File: elisp,  Node: Syntax of Regexps,  Next: Regexp Example,  Up: Regular Expressions

34.3.1 Syntax of Regular Expressions
------------------------------------

Regular expressions have a syntax in which a few characters are special
constructs and the rest are "ordinary".  An ordinary character is a
simple regular expression that matches that character and nothing else.
The special characters are `.', `*', `+', `?', `[', `^', `$', and `\';
no new special characters will be defined in the future.  The character
`]' is special if it ends a character alternative (see later).  The
character `-' is special inside a character alternative.  A `[:' and
balancing `:]' enclose a character class inside a character
alternative.  Any other character appearing in a regular expression is
ordinary, unless a `\' precedes it.

   For example, `f' is not a special character, so it is ordinary, and
therefore `f' is a regular expression that matches the string `f' and
no other string.  (It does _not_ match the string `fg', but it does
match a _part_ of that string.)  Likewise, `o' is a regular expression
that matches only `o'.

   Any two regular expressions A and B can be concatenated.  The result
is a regular expression that matches a string if A matches some amount
of the beginning of that string and B matches the rest of the string.

   As a simple example, we can concatenate the regular expressions `f'
and `o' to get the regular expression `fo', which matches only the
string `fo'.  Still trivial.  To do something more powerful, you need
to use one of the special regular expression constructs.

* Menu:

* Regexp Special::      Special characters in regular expressions.
* Char Classes::        Character classes used in regular expressions.
* Regexp Backslash::    Backslash-sequences in regular expressions.


File: elisp,  Node: Regexp Special,  Next: Char Classes,  Up: Syntax of Regexps

34.3.1.1 Special Characters in Regular Expressions
..................................................

Here is a list of the characters that are special in a regular
expression.

`.' (Period)
     is a special character that matches any single character except a
     newline.  Using concatenation, we can make regular expressions
     like `a.b', which matches any three-character string that begins
     with `a' and ends with `b'.

`*'
     is not a construct by itself; it is a postfix operator that means
     to match the preceding regular expression repetitively as many
     times as possible.  Thus, `o*' matches any number of `o's
     (including no `o's).

     `*' always applies to the _smallest_ possible preceding
     expression.  Thus, `fo*' has a repeating `o', not a repeating
     `fo'.  It matches `f', `fo', `foo', and so on.

     The matcher processes a `*' construct by matching, immediately, as
     many repetitions as can be found.  Then it continues with the rest
     of the pattern.  If that fails, backtracking occurs, discarding
     some of the matches of the `*'-modified construct in the hope that
     that will make it possible to match the rest of the pattern.  For
     example, in matching `ca*ar' against the string `caaar', the `a*'
     first tries to match all three `a's; but the rest of the pattern is
     `ar' and there is only `r' left to match, so this try fails.  The
     next alternative is for `a*' to match only two `a's.  With this
     choice, the rest of the regexp matches successfully.

     *Warning:* Nested repetition operators can run for an indefinitely
     long time, if they lead to ambiguous matching.  For example,
     trying to match the regular expression `\(x+y*\)*a' against the
     string `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxz' could take hours
     before it ultimately fails.  Emacs must try each way of grouping
     the `x's before concluding that none of them can work.  Even
     worse, `\(x*\)*' can match the null string in infinitely many
     ways, so it causes an infinite loop.  To avoid these problems,
     check nested repetitions carefully, to make sure that they do not
     cause combinatorial explosions in backtracking.

`+'
     is a postfix operator, similar to `*' except that it must match
     the preceding expression at least once.  So, for example, `ca+r'
     matches the strings `car' and `caaaar' but not the string `cr',
     whereas `ca*r' matches all three strings.

`?'
     is a postfix operator, similar to `*' except that it must match the
     preceding expression either once or not at all.  For example,
     `ca?r' matches `car' or `cr'; nothing else.

`*?', `+?', `??'
     These are "non-greedy" variants of the operators `*', `+' and `?'.
     Where those operators match the largest possible substring
     (consistent with matching the entire containing expression), the
     non-greedy variants match the smallest possible substring
     (consistent with matching the entire containing expression).

     For example, the regular expression `c[ad]*a' when applied to the
     string `cdaaada' matches the whole string; but the regular
     expression `c[ad]*?a', applied to that same string, matches just
     `cda'.  (The smallest possible match here for `[ad]*?' that
     permits the whole expression to match is `d'.)

`[ ... ]'
     is a "character alternative", which begins with `[' and is
     terminated by `]'.  In the simplest case, the characters between
     the two brackets are what this character alternative can match.

     Thus, `[ad]' matches either one `a' or one `d', and `[ad]*'
     matches any string composed of just `a's and `d's (including the
     empty string).  It follows that `c[ad]*r' matches `cr', `car',
     `cdr', `caddaar', etc.

     You can also include character ranges in a character alternative,
     by writing the starting and ending characters with a `-' between
     them.  Thus, `[a-z]' matches any lower-case ASCII letter.  Ranges
     may be intermixed freely with individual characters, as in
     `[a-z$%.]', which matches any lower case ASCII letter or `$', `%'
     or period.

     If `case-fold-search' is non-`nil', `[a-z]' also matches
     upper-case letters.  Note that a range like `[a-z]' is not
     affected by the locale's collation sequence, it always represents
     a sequence in ASCII order.

     Note also that the usual regexp special characters are not special
     inside a character alternative.  A completely different set of
     characters is special inside character alternatives: `]', `-' and
     `^'.

     To include a `]' in a character alternative, you must make it the
     first character.  For example, `[]a]' matches `]' or `a'.  To
     include a `-', write `-' as the first or last character of the
     character alternative, or put it after a range.  Thus, `[]-]'
     matches both `]' and `-'.  (As explained below, you cannot use
     `\]' to include a `]' inside a character alternative, since `\' is
     not special there.)

     To include `^' in a character alternative, put it anywhere but at
     the beginning.

     If a range starts with a unibyte character C and ends with a
     multibyte character C2, the range is divided into two parts: one
     spans the unibyte characters `C..?\377', the other the multibyte
     characters `C1..C2', where C1 is the first character of the
     charset to which C2 belongs.

     A character alternative can also specify named character classes
     (*note Char Classes::).  This is a POSIX feature.  For example,
     `[[:ascii:]]' matches any ASCII character.  Using a character
     class is equivalent to mentioning each of the characters in that
     class; but the latter is not feasible in practice, since some
     classes include thousands of different characters.

`[^ ... ]'
     `[^' begins a "complemented character alternative".  This matches
     any character except the ones specified.  Thus, `[^a-z0-9A-Z]'
     matches all characters _except_ letters and digits.

     `^' is not special in a character alternative unless it is the
     first character.  The character following the `^' is treated as if
     it were first (in other words, `-' and `]' are not special there).

     A complemented character alternative can match a newline, unless
     newline is mentioned as one of the characters not to match.  This
     is in contrast to the handling of regexps in programs such as
     `grep'.

     You can specify named character classes, just like in character
     alternatives.  For instance, `[^[:ascii:]]' matches any non-ASCII
     character.  *Note Char Classes::.

`^'
     When matching a buffer, `^' matches the empty string, but only at
     the beginning of a line in the text being matched (or the
     beginning of the accessible portion of the buffer).  Otherwise it
     fails to match anything.  Thus, `^foo' matches a `foo' that occurs
     at the beginning of a line.

     When matching a string instead of a buffer, `^' matches at the
     beginning of the string or after a newline character.

     For historical compatibility reasons, `^' can be used only at the
     beginning of the regular expression, or after `\(', `\(?:' or `\|'.

`$'
     is similar to `^' but matches only at the end of a line (or the
     end of the accessible portion of the buffer).  Thus, `x+$' matches
     a string of one `x' or more at the end of a line.

     When matching a string instead of a buffer, `$' matches at the end
     of the string or before a newline character.

     For historical compatibility reasons, `$' can be used only at the
     end of the regular expression, or before `\)' or `\|'.

`\'
     has two functions: it quotes the special characters (including
     `\'), and it introduces additional special constructs.

     Because `\' quotes special characters, `\$' is a regular
     expression that matches only `$', and `\[' is a regular expression
     that matches only `[', and so on.

     Note that `\' also has special meaning in the read syntax of Lisp
     strings (*note String Type::), and must be quoted with `\'.  For
     example, the regular expression that matches the `\' character is
     `\\'.  To write a Lisp string that contains the characters `\\',
     Lisp syntax requires you to quote each `\' with another `\'.
     Therefore, the read syntax for a regular expression matching `\'
     is `"\\\\"'.

   *Please note:* For historical compatibility, special characters are
treated as ordinary ones if they are in contexts where their special
meanings make no sense.  For example, `*foo' treats `*' as ordinary
since there is no preceding expression on which the `*' can act.  It is
poor practice to depend on this behavior; quote the special character
anyway, regardless of where it appears.

   As a `\' is not special inside a character alternative, it can never
remove the special meaning of `-' or `]'.  So you should not quote
these characters when they have no special meaning either.  This would
not clarify anything, since backslashes can legitimately precede these
characters where they _have_ special meaning, as in `[^\]' (`"[^\\]"'
for Lisp string syntax), which matches any single character except a
backslash.

   In practice, most `]' that occur in regular expressions close a
character alternative and hence are special.  However, occasionally a
regular expression may try to match a complex pattern of literal `['
and `]'.  In such situations, it sometimes may be necessary to
carefully parse the regexp from the start to determine which square
brackets enclose a character alternative.  For example, `[^][]]'
consists of the complemented character alternative `[^][]' (which
matches any single character that is not a square bracket), followed by
a literal `]'.

   The exact rules are that at the beginning of a regexp, `[' is
special and `]' not.  This lasts until the first unquoted `[', after
which we are in a character alternative; `[' is no longer special
(except when it starts a character class) but `]' is special, unless it
immediately follows the special `[' or that `[' followed by a `^'.
This lasts until the next special `]' that does not end a character
class.  This ends the character alternative and restores the ordinary
syntax of regular expressions; an unquoted `[' is special again and a
`]' not.


File: elisp,  Node: Char Classes,  Next: Regexp Backslash,  Prev: Regexp Special,  Up: Syntax of Regexps

34.3.1.2 Character Classes
..........................

Here is a table of the classes you can use in a character alternative,
and what they mean:

`[:ascii:]'
     This matches any ASCII character (codes 0-127).

`[:alnum:]'
     This matches any letter or digit.  (At present, for multibyte
     characters, it matches anything that has word syntax.)

`[:alpha:]'
     This matches any letter.  (At present, for multibyte characters, it
     matches anything that has word syntax.)

`[:blank:]'
     This matches space and tab only.

`[:cntrl:]'
     This matches any ASCII control character.

`[:digit:]'
     This matches `0' through `9'.  Thus, `[-+[:digit:]]' matches any
     digit, as well as `+' and `-'.

`[:graph:]'
     This matches graphic characters--everything except ASCII control
     characters, space, and the delete character.

`[:lower:]'
     This matches any lower-case letter, as determined by the current
     case table (*note Case Tables::).  If `case-fold-search' is
     non-`nil', this also matches any upper-case letter.

`[:multibyte:]'
     This matches any multibyte character (*note Text
     Representations::).

`[:nonascii:]'
     This matches any non-ASCII character.

`[:print:]'
     This matches printing characters--everything except ASCII control
     characters and the delete character.

`[:punct:]'
     This matches any punctuation character.  (At present, for multibyte
     characters, it matches anything that has non-word syntax.)

`[:space:]'
     This matches any character that has whitespace syntax (*note
     Syntax Class Table::).

`[:unibyte:]'
     This matches any unibyte character (*note Text Representations::).

`[:upper:]'
     This matches any upper-case letter, as determined by the current
     case table (*note Case Tables::).  If `case-fold-search' is
     non-`nil', this also matches any lower-case letter.

`[:word:]'
     This matches any character that has word syntax (*note Syntax
     Class Table::).

`[:xdigit:]'
     This matches the hexadecimal digits: `0' through `9', `a' through
     `f' and `A' through `F'.


File: elisp,  Node: Regexp Backslash,  Prev: Char Classes,  Up: Syntax of Regexps

34.3.1.3 Backslash Constructs in Regular Expressions
....................................................

For the most part, `\' followed by any character matches only that
character.  However, there are several exceptions: certain
two-character sequences starting with `\' that have special meanings.
(The character after the `\' in such a sequence is always ordinary when
used on its own.)  Here is a table of the special `\' constructs.

`\|'
     specifies an alternative.  Two regular expressions A and B with
     `\|' in between form an expression that matches anything that
     either A or B matches.

     Thus, `foo\|bar' matches either `foo' or `bar' but no other string.

     `\|' applies to the largest possible surrounding expressions.
     Only a surrounding `\( ... \)' grouping can limit the grouping
     power of `\|'.

     If you need full backtracking capability to handle multiple uses of
     `\|', use the POSIX regular expression functions (*note POSIX
     Regexps::).

`\{M\}'
     is a postfix operator that repeats the previous pattern exactly M
     times.  Thus, `x\{5\}' matches the string `xxxxx' and nothing
     else.  `c[ad]\{3\}r' matches string such as `caaar', `cdddr',
     `cadar', and so on.

`\{M,N\}'
     is a more general postfix operator that specifies repetition with a
     minimum of M repeats and a maximum of N repeats.  If M is omitted,
     the minimum is 0; if N is omitted, there is no maximum.

     For example, `c[ad]\{1,2\}r' matches the strings `car', `cdr',
     `caar', `cadr', `cdar', and `cddr', and nothing else.
     `\{0,1\}' or `\{,1\}' is equivalent to `?'.
     `\{0,\}' or `\{,\}' is equivalent to `*'.
     `\{1,\}' is equivalent to `+'.

`\( ... \)'
     is a grouping construct that serves three purposes:

       1. To enclose a set of `\|' alternatives for other operations.
          Thus, the regular expression `\(foo\|bar\)x' matches either
          `foox' or `barx'.

       2. To enclose a complicated expression for the postfix operators
          `*', `+' and `?' to operate on.  Thus, `ba\(na\)*' matches
          `ba', `bana', `banana', `bananana', etc., with any number
          (zero or more) of `na' strings.

       3. To record a matched substring for future reference with
          `\DIGIT' (see below).

     This last application is not a consequence of the idea of a
     parenthetical grouping; it is a separate feature that was assigned
     as a second meaning to the same `\( ... \)' construct because, in
     practice, there was usually no conflict between the two meanings.
     But occasionally there is a conflict, and that led to the
     introduction of shy groups.

`\(?: ... \)'
     is the "shy group" construct.  A shy group serves the first two
     purposes of an ordinary group (controlling the nesting of other
     operators), but it does not get a number, so you cannot refer back
     to its value with `\DIGIT'.  Shy groups are particularly useful
     for mechanically-constructed regular expressions, because they can
     be added automatically without altering the numbering of ordinary,
     non-shy groups.

     Shy groups are also called "non-capturing" or "unnumbered groups".

`\(?NUM: ... \)'
     is the "explicitly numbered group" construct.  Normal groups get
     their number implicitly, based on their position, which can be
     inconvenient.  This construct allows you to force a particular
     group number.  There is no particular restriction on the numbering,
     e.g. you can have several groups with the same number in which case
     the last one to match (i.e. the rightmost match) will win.
     Implicitly numbered groups always get the smallest integer larger
     than the one of any previous group.

`\DIGIT'
     matches the same text that matched the DIGITth occurrence of a
     grouping (`\( ... \)') construct.

     In other words, after the end of a group, the matcher remembers the
     beginning and end of the text matched by that group.  Later on in
     the regular expression you can use `\' followed by DIGIT to match
     that same text, whatever it may have been.

     The strings matching the first nine grouping constructs appearing
     in the entire regular expression passed to a search or matching
     function are assigned numbers 1 through 9 in the order that the
     open parentheses appear in the regular expression.  So you can use
     `\1' through `\9' to refer to the text matched by the
     corresponding grouping constructs.

     For example, `\(.*\)\1' matches any newline-free string that is
     composed of two identical halves.  The `\(.*\)' matches the first
     half, which may be anything, but the `\1' that follows must match
     the same exact text.

     If a `\( ... \)' construct matches more than once (which can
     happen, for instance, if it is followed by `*'), only the last
     match is recorded.

     If a particular grouping construct in the regular expression was
     never matched--for instance, if it appears inside of an
     alternative that wasn't used, or inside of a repetition that
     repeated zero times--then the corresponding `\DIGIT' construct
     never matches anything.  To use an artificial example,
     `\(foo\(b*\)\|lose\)\2' cannot match `lose': the second
     alternative inside the larger group matches it, but then `\2' is
     undefined and can't match anything.  But it can match `foobb',
     because the first alternative matches `foob' and `\2' matches `b'.

`\w'
     matches any word-constituent character.  The editor syntax table
     determines which characters these are.  *Note Syntax Tables::.

`\W'
     matches any character that is not a word constituent.

`\sCODE'
     matches any character whose syntax is CODE.  Here CODE is a
     character that represents a syntax code: thus, `w' for word
     constituent, `-' for whitespace, `(' for open parenthesis, etc.
     To represent whitespace syntax, use either `-' or a space
     character.  *Note Syntax Class Table::, for a list of syntax codes
     and the characters that stand for them.

`\SCODE'
     matches any character whose syntax is not CODE.

`\cC'
     matches any character whose category is C.  Here C is a character
     that represents a category: thus, `c' for Chinese characters or
     `g' for Greek characters in the standard category table.  You can
     see the list of all the currently defined categories with `M-x
     describe-categories <RET>'.  You can also define your own
     categories in addition to the standard ones using the
     `define-category' function (*note Categories::).

`\CC'
     matches any character whose category is not C.

   The following regular expression constructs match the empty
string--that is, they don't use up any characters--but whether they
match depends on the context.  For all, the beginning and end of the
accessible portion of the buffer are treated as if they were the actual
beginning and end of the buffer.

`\`'
     matches the empty string, but only at the beginning of the buffer
     or string being matched against.

`\''
     matches the empty string, but only at the end of the buffer or
     string being matched against.

`\='
     matches the empty string, but only at point.  (This construct is
     not defined when matching against a string.)

`\b'
     matches the empty string, but only at the beginning or end of a
     word.  Thus, `\bfoo\b' matches any occurrence of `foo' as a
     separate word.  `\bballs?\b' matches `ball' or `balls' as a
     separate word.

     `\b' matches at the beginning or end of the buffer (or string)
     regardless of what text appears next to it.

`\B'
     matches the empty string, but _not_ at the beginning or end of a
     word, nor at the beginning or end of the buffer (or string).

`\<'
     matches the empty string, but only at the beginning of a word.
     `\<' matches at the beginning of the buffer (or string) only if a
     word-constituent character follows.

`\>'
     matches the empty string, but only at the end of a word.  `\>'
     matches at the end of the buffer (or string) only if the contents
     end with a word-constituent character.

`\_<'
     matches the empty string, but only at the beginning of a symbol.  A
     symbol is a sequence of one or more word or symbol constituent
     characters.  `\_<' matches at the beginning of the buffer (or
     string) only if a symbol-constituent character follows.

`\_>'
     matches the empty string, but only at the end of a symbol.  `\_>'
     matches at the end of the buffer (or string) only if the contents
     end with a symbol-constituent character.

   Not every string is a valid regular expression.  For example, a
string that ends inside a character alternative without a terminating
`]' is invalid, and so is a string that ends with a single `\'.  If an
invalid regular expression is passed to any of the search functions, an
`invalid-regexp' error is signaled.


File: elisp,  Node: Regexp Example,  Next: Regexp Functions,  Prev: Syntax of Regexps,  Up: Regular Expressions

34.3.2 Complex Regexp Example
-----------------------------

Here is a complicated regexp which was formerly used by Emacs to
recognize the end of a sentence together with any whitespace that
follows.  (Nowadays Emacs uses a similar but more complex default
regexp constructed by the function `sentence-end'.  *Note Standard
Regexps::.)

   Below, we show first the regexp as a string in Lisp syntax (to
distinguish spaces from tab characters), and then the result of
evaluating it.  The string constant begins and ends with a
double-quote.  `\"' stands for a double-quote as part of the string,
`\\' for a backslash as part of the string, `\t' for a tab and `\n' for
a newline.

     "[.?!][]\"')}]*\\($\\| $\\|\t\\|  \\)[ \t\n]*"
          => "[.?!][]\"')}]*\\($\\| $\\|  \\|  \\)[
     ]*"

In the output, tab and newline appear as themselves.

   This regular expression contains four parts in succession and can be
deciphered as follows:

`[.?!]'
     The first part of the pattern is a character alternative that
     matches any one of three characters: period, question mark, and
     exclamation mark.  The match must begin with one of these three
     characters.  (This is one point where the new default regexp used
     by Emacs differs from the old.  The new value also allows some
     non-ASCII characters that end a sentence without any following
     whitespace.)

`[]\"')}]*'
     The second part of the pattern matches any closing braces and
     quotation marks, zero or more of them, that may follow the period,
     question mark or exclamation mark.  The `\"' is Lisp syntax for a
     double-quote in a string.  The `*' at the end indicates that the
     immediately preceding regular expression (a character alternative,
     in this case) may be repeated zero or more times.

`\\($\\| $\\|\t\\|  \\)'
     The third part of the pattern matches the whitespace that follows
     the end of a sentence: the end of a line (optionally with a
     space), or a tab, or two spaces.  The double backslashes mark the
     parentheses and vertical bars as regular expression syntax; the
     parentheses delimit a group and the vertical bars separate
     alternatives.  The dollar sign is used to match the end of a line.

`[ \t\n]*'
     Finally, the last part of the pattern matches any additional
     whitespace beyond the minimum needed to end a sentence.


File: elisp,  Node: Regexp Functions,  Prev: Regexp Example,  Up: Regular Expressions

34.3.3 Regular Expression Functions
-----------------------------------

These functions operate on regular expressions.

 -- Function: regexp-quote string
     This function returns a regular expression whose only exact match
     is STRING.  Using this regular expression in `looking-at' will
     succeed only if the next characters in the buffer are STRING;
     using it in a search function will succeed if the text being
     searched contains STRING.  *Note Regexp Search::.

     This allows you to request an exact string match or search when
     calling a function that wants a regular expression.

          (regexp-quote "^The cat$")
               => "\\^The cat\\$"

     One use of `regexp-quote' is to combine an exact string match with
     context described as a regular expression.  For example, this
     searches for the string that is the value of STRING, surrounded by
     whitespace:

          (re-search-forward
           (concat "\\s-" (regexp-quote string) "\\s-"))

 -- Function: regexp-opt strings &optional paren
     This function returns an efficient regular expression that will
     match any of the strings in the list STRINGS.  This is useful when
     you need to make matching or searching as fast as possible--for
     example, for Font Lock mode(1).

     If the optional argument PAREN is non-`nil', then the returned
     regular expression is always enclosed by at least one
     parentheses-grouping construct.  If PAREN is `words', then that
     construct is additionally surrounded by `\<' and `\>';
     alternatively, if PAREN is `symbols', then that construct is
     additionally surrounded by `\_<' and `\_>' (`symbols' is often
     appropriate when matching programming-language keywords and the
     like).

     This simplified definition of `regexp-opt' produces a regular
     expression which is equivalent to the actual value (but not as
     efficient):

          (defun regexp-opt (strings &optional paren)
            (let ((open-paren (if paren "\\(" ""))
                  (close-paren (if paren "\\)" "")))
              (concat open-paren
                      (mapconcat 'regexp-quote strings "\\|")
                      close-paren)))

 -- Function: regexp-opt-depth regexp
     This function returns the total number of grouping constructs
     (parenthesized expressions) in REGEXP.  This does not include shy
     groups (*note Regexp Backslash::).

 -- Function: regexp-opt-charset chars
     This function returns a regular expression matching a character in
     the list of characters CHARS.

          (regexp-opt-charset '(?a ?b ?c ?d ?e))
               => "[a-e]"

   ---------- Footnotes ----------

   (1) Note that `regexp-opt' does not guarantee that its result is
absolutely the most efficient form possible.  A hand-tuned regular
expression can sometimes be slightly more efficient, but is almost
never worth the effort.


File: elisp,  Node: Regexp Search,  Next: POSIX Regexps,  Prev: Regular Expressions,  Up: Searching and Matching

34.4 Regular Expression Searching
=================================

In GNU Emacs, you can search for the next match for a regular
expression either incrementally or not.  For incremental search
commands, see *note Regular Expression Search: (emacs)Regexp Search.
Here we describe only the search functions useful in programs.  The
principal one is `re-search-forward'.

   These search functions convert the regular expression to multibyte if
the buffer is multibyte; they convert the regular expression to unibyte
if the buffer is unibyte.  *Note Text Representations::.

 -- Command: re-search-forward regexp &optional limit noerror repeat
     This function searches forward in the current buffer for a string
     of text that is matched by the regular expression REGEXP.  The
     function skips over any amount of text that is not matched by
     REGEXP, and leaves point at the end of the first match found.  It
     returns the new value of point.

     If LIMIT is non-`nil', it must be a position in the current
     buffer.  It specifies the upper bound to the search.  No match
     extending after that position is accepted.

     If REPEAT is supplied, it must be a positive number; the search is
     repeated that many times; each repetition starts at the end of the
     previous match.  If all these successive searches succeed, the
     search succeeds, moving point and returning its new value.
     Otherwise the search fails.  What `re-search-forward' does when
     the search fails depends on the value of NOERROR:

    `nil'
          Signal a `search-failed' error.

    `t'
          Do nothing and return `nil'.

    anything else
          Move point to LIMIT (or the end of the accessible portion of
          the buffer) and return `nil'.

     In the following example, point is initially before the `T'.
     Evaluating the search call moves point to the end of that line
     (between the `t' of `hat' and the newline).

          ---------- Buffer: foo ----------
          I read "-!-The cat in the hat
          comes back" twice.
          ---------- Buffer: foo ----------

          (re-search-forward "[a-z]+" nil t 5)
               => 27

          ---------- Buffer: foo ----------
          I read "The cat in the hat-!-
          comes back" twice.
          ---------- Buffer: foo ----------

 -- Command: re-search-backward regexp &optional limit noerror repeat
     This function searches backward in the current buffer for a string
     of text that is matched by the regular expression REGEXP, leaving
     point at the beginning of the first text found.

     This function is analogous to `re-search-forward', but they are not
     simple mirror images.  `re-search-forward' finds the match whose
     beginning is as close as possible to the starting point.  If
     `re-search-backward' were a perfect mirror image, it would find the
     match whose end is as close as possible.  However, in fact it
     finds the match whose beginning is as close as possible (and yet
     ends before the starting point).  The reason for this is that
     matching a regular expression at a given spot always works from
     beginning to end, and starts at a specified beginning position.

     A true mirror-image of `re-search-forward' would require a special
     feature for matching regular expressions from end to beginning.
     It's not worth the trouble of implementing that.

 -- Function: string-match regexp string &optional start
     This function returns the index of the start of the first match for
     the regular expression REGEXP in STRING, or `nil' if there is no
     match.  If START is non-`nil', the search starts at that index in
     STRING.

     For example,

          (string-match
           "quick" "The quick brown fox jumped quickly.")
               => 4
          (string-match
           "quick" "The quick brown fox jumped quickly." 8)
               => 27

     The index of the first character of the string is 0, the index of
     the second character is 1, and so on.

     After this function returns, the index of the first character
     beyond the match is available as `(match-end 0)'.  *Note Match
     Data::.

          (string-match
           "quick" "The quick brown fox jumped quickly." 8)
               => 27

          (match-end 0)
               => 32

 -- Function: string-match-p regexp string &optional start
     This predicate function does what `string-match' does, but it
     avoids modifying the match data.

 -- Function: looking-at regexp
     This function determines whether the text in the current buffer
     directly following point matches the regular expression REGEXP.
     "Directly following" means precisely that: the search is
     "anchored" and it can succeed only starting with the first
     character following point.  The result is `t' if so, `nil'
     otherwise.

     This function does not move point, but it does update the match
     data.  *Note Match Data::.  If you need to test for a match
     without modifying the match data, use `looking-at-p', described
     below.

     In this example, point is located directly before the `T'.  If it
     were anywhere else, the result would be `nil'.

          ---------- Buffer: foo ----------
          I read "-!-The cat in the hat
          comes back" twice.
          ---------- Buffer: foo ----------

          (looking-at "The cat in the hat$")
               => t

 -- Function: looking-back regexp &optional limit greedy
     This function returns `t' if REGEXP matches the text immediately
     before point (i.e., ending at point), and `nil' otherwise.

     Because regular expression matching works only going forward, this
     is implemented by searching backwards from point for a match that
     ends at point.  That can be quite slow if it has to search a long
     distance.  You can bound the time required by specifying LIMIT,
     which says not to search before LIMIT.  In this case, the match
     that is found must begin at or after LIMIT.

     If GREEDY is non-`nil', this function extends the match backwards
     as far as possible, stopping when a single additional previous
     character cannot be part of a match for regexp.  When the match is
     extended, its starting position is allowed to occur before LIMIT.

          ---------- Buffer: foo ----------
          I read "-!-The cat in the hat
          comes back" twice.
          ---------- Buffer: foo ----------

          (looking-back "read \"" 3)
               => t
          (looking-back "read \"" 4)
               => nil

     As a general recommendation, try to avoid using `looking-back'
     wherever possible, since it is slow.  For this reason, there are no
     plans to add a `looking-back-p' function.

 -- Function: looking-at-p regexp
     This predicate function works like `looking-at', but without
     updating the match data.

 -- Variable: search-spaces-regexp
     If this variable is non-`nil', it should be a regular expression
     that says how to search for whitespace.  In that case, any group of
     spaces in a regular expression being searched for stands for use of
     this regular expression.  However, spaces inside of constructs
     such as `[...]' and `*', `+', `?' are not affected by
     `search-spaces-regexp'.

     Since this variable affects all regular expression search and match
     constructs, you should bind it temporarily for as small as possible
     a part of the code.


File: elisp,  Node: POSIX Regexps,  Next: Match Data,  Prev: Regexp Search,  Up: Searching and Matching

34.5 POSIX Regular Expression Searching
=======================================

The usual regular expression functions do backtracking when necessary
to handle the `\|' and repetition constructs, but they continue this
only until they find _some_ match.  Then they succeed and report the
first match found.

   This section describes alternative search functions which perform the
full backtracking specified by the POSIX standard for regular expression
matching.  They continue backtracking until they have tried all
possibilities and found all matches, so they can report the longest
match, as required by POSIX.  This is much slower, so use these
functions only when you really need the longest match.

   The POSIX search and match functions do not properly support the
non-greedy repetition operators (*note non-greedy: Regexp Special.).
This is because POSIX backtracking conflicts with the semantics of
non-greedy repetition.

 -- Command: posix-search-forward regexp &optional limit noerror repeat
     This is like `re-search-forward' except that it performs the full
     backtracking specified by the POSIX standard for regular expression
     matching.

 -- Command: posix-search-backward regexp &optional limit noerror repeat
     This is like `re-search-backward' except that it performs the full
     backtracking specified by the POSIX standard for regular expression
     matching.

 -- Function: posix-looking-at regexp
     This is like `looking-at' except that it performs the full
     backtracking specified by the POSIX standard for regular expression
     matching.

 -- Function: posix-string-match regexp string &optional start
     This is like `string-match' except that it performs the full
     backtracking specified by the POSIX standard for regular expression
     matching.


File: elisp,  Node: Match Data,  Next: Search and Replace,  Prev: POSIX Regexps,  Up: Searching and Matching

34.6 The Match Data
===================

Emacs keeps track of the start and end positions of the segments of
text found during a search; this is called the "match data".  Thanks to
the match data, you can search for a complex pattern, such as a date in
a mail message, and then extract parts of the match under control of
the pattern.

   Because the match data normally describe the most recent search only,
you must be careful not to do another search inadvertently between the
search you wish to refer back to and the use of the match data.  If you
can't avoid another intervening search, you must save and restore the
match data around it, to prevent it from being overwritten.

   Notice that all functions are allowed to overwrite the match data
unless they're explicitly documented not to do so.  A consequence is
that functions that are run implicitly in the background (*note
Timers::, and *note Idle Timers::) should likely save and restore the
match data explicitly.

* Menu:

* Replacing Match::       Replacing a substring that was matched.
* Simple Match Data::     Accessing single items of match data,
                            such as where a particular subexpression started.
* Entire Match Data::     Accessing the entire match data at once, as a list.
* Saving Match Data::     Saving and restoring the match data.


File: elisp,  Node: Replacing Match,  Next: Simple Match Data,  Up: Match Data

34.6.1 Replacing the Text that Matched
--------------------------------------

This function replaces all or part of the text matched by the last
search.  It works by means of the match data.

 -- Function: replace-match replacement &optional fixedcase literal
          string subexp
     This function replaces the text in the buffer (or in STRING) that
     was matched by the last search.  It replaces that text with
     REPLACEMENT.

     If you did the last search in a buffer, you should specify `nil'
     for STRING and make sure that the current buffer when you call
     `replace-match' is the one in which you did the searching or
     matching.  Then `replace-match' does the replacement by editing
     the buffer; it leaves point at the end of the replacement text, and
     returns `t'.

     If you did the search in a string, pass the same string as STRING.
     Then `replace-match' does the replacement by constructing and
     returning a new string.

     If FIXEDCASE is non-`nil', then `replace-match' uses the
     replacement text without case conversion; otherwise, it converts
     the replacement text depending upon the capitalization of the text
     to be replaced.  If the original text is all upper case, this
     converts the replacement text to upper case.  If all words of the
     original text are capitalized, this capitalizes all the words of
     the replacement text.  If all the words are one-letter and they
     are all upper case, they are treated as capitalized words rather
     than all-upper-case words.

     If LITERAL is non-`nil', then REPLACEMENT is inserted exactly as
     it is, the only alterations being case changes as needed.  If it
     is `nil' (the default), then the character `\' is treated
     specially.  If a `\' appears in REPLACEMENT, then it must be part
     of one of the following sequences:

    `\&'
          `\&' stands for the entire text being replaced.

    `\N'
          `\N', where N is a digit, stands for the text that matched
          the Nth subexpression in the original regexp.  Subexpressions
          are those expressions grouped inside `\(...\)'.  If the Nth
          subexpression never matched, an empty string is substituted.

    `\\'
          `\\' stands for a single `\' in the replacement text.

     These substitutions occur after case conversion, if any, so the
     strings they substitute are never case-converted.

     If SUBEXP is non-`nil', that says to replace just subexpression
     number SUBEXP of the regexp that was matched, not the entire
     match.  For example, after matching `foo \(ba*r\)', calling
     `replace-match' with 1 as SUBEXP means to replace just the text
     that matched `\(ba*r\)'.

 -- Function: match-substitute-replacement replacement &optional
          fixedcase literal string subexp
     This function returns the text that would be inserted into the
     buffer by `replace-match', but without modifying the buffer.  It is
     useful if you want to present the user with actual replacement
     result, with constructs like `\N' or `\&' substituted with matched
     groups.  Arguments REPLACEMENT and optional FIXEDCASE, LITERAL,
     STRING and SUBEXP have the same meaning as for `replace-match'.


File: elisp,  Node: Simple Match Data,  Next: Entire Match Data,  Prev: Replacing Match,  Up: Match Data

34.6.2 Simple Match Data Access
-------------------------------

This section explains how to use the match data to find out what was
matched by the last search or match operation, if it succeeded.

   You can ask about the entire matching text, or about a particular
parenthetical subexpression of a regular expression.  The COUNT
argument in the functions below specifies which.  If COUNT is zero, you
are asking about the entire match.  If COUNT is positive, it specifies
which subexpression you want.

   Recall that the subexpressions of a regular expression are those
expressions grouped with escaped parentheses, `\(...\)'.  The COUNTth
subexpression is found by counting occurrences of `\(' from the
beginning of the whole regular expression.  The first subexpression is
numbered 1, the second 2, and so on.  Only regular expressions can have
subexpressions--after a simple string search, the only information
available is about the entire match.

   Every successful search sets the match data.  Therefore, you should
query the match data immediately after searching, before calling any
other function that might perform another search.  Alternatively, you
may save and restore the match data (*note Saving Match Data::) around
the call to functions that could perform another search.  Or use the
functions that explicitly do not modify the match data; e.g.
`string-match-p'.

   A search which fails may or may not alter the match data.  In the
current implementation, it does not, but we may change it in the
future.  Don't try to rely on the value of the match data after a
failing search.

 -- Function: match-string count &optional in-string
     This function returns, as a string, the text matched in the last
     search or match operation.  It returns the entire text if COUNT is
     zero, or just the portion corresponding to the COUNTth
     parenthetical subexpression, if COUNT is positive.

     If the last such operation was done against a string with
     `string-match', then you should pass the same string as the
     argument IN-STRING.  After a buffer search or match, you should
     omit IN-STRING or pass `nil' for it; but you should make sure that
     the current buffer when you call `match-string' is the one in
     which you did the searching or matching.  Failure to follow this
     advice will lead to incorrect results.

     The value is `nil' if COUNT is out of range, or for a
     subexpression inside a `\|' alternative that wasn't used or a
     repetition that repeated zero times.

 -- Function: match-string-no-properties count &optional in-string
     This function is like `match-string' except that the result has no
     text properties.

 -- Function: match-beginning count
     This function returns the position of the start of the text
     matched by the last regular expression searched for, or a
     subexpression of it.

     If COUNT is zero, then the value is the position of the start of
     the entire match.  Otherwise, COUNT specifies a subexpression in
     the regular expression, and the value of the function is the
     starting position of the match for that subexpression.

     The value is `nil' for a subexpression inside a `\|' alternative
     that wasn't used or a repetition that repeated zero times.

 -- Function: match-end count
     This function is like `match-beginning' except that it returns the
     position of the end of the match, rather than the position of the
     beginning.

   Here is an example of using the match data, with a comment showing
the positions within the text:

     (string-match "\\(qu\\)\\(ick\\)"
                   "The quick fox jumped quickly.")
                   ;0123456789
          => 4

     (match-string 0 "The quick fox jumped quickly.")
          => "quick"
     (match-string 1 "The quick fox jumped quickly.")
          => "qu"
     (match-string 2 "The quick fox jumped quickly.")
          => "ick"

     (match-beginning 1)       ; The beginning of the match
          => 4                 ;   with `qu' is at index 4.

     (match-beginning 2)       ; The beginning of the match
          => 6                 ;   with `ick' is at index 6.

     (match-end 1)             ; The end of the match
          => 6                 ;   with `qu' is at index 6.

     (match-end 2)             ; The end of the match
          => 9                 ;   with `ick' is at index 9.

   Here is another example.  Point is initially located at the beginning
of the line.  Searching moves point to between the space and the word
`in'.  The beginning of the entire match is at the 9th character of the
buffer (`T'), and the beginning of the match for the first
subexpression is at the 13th character (`c').

     (list
       (re-search-forward "The \\(cat \\)")
       (match-beginning 0)
       (match-beginning 1))
         => (17 9 13)

     ---------- Buffer: foo ----------
     I read "The cat -!-in the hat comes back" twice.
             ^   ^
             9  13
     ---------- Buffer: foo ----------

(In this case, the index returned is a buffer position; the first
character of the buffer counts as 1.)


File: elisp,  Node: Entire Match Data,  Next: Saving Match Data,  Prev: Simple Match Data,  Up: Match Data

34.6.3 Accessing the Entire Match Data
--------------------------------------

The functions `match-data' and `set-match-data' read or write the
entire match data, all at once.

 -- Function: match-data &optional integers reuse reseat
     This function returns a list of positions (markers or integers)
     that record all the information on the text that the last search
     matched.  Element zero is the position of the beginning of the
     match for the whole expression; element one is the position of the
     end of the match for the expression.  The next two elements are
     the positions of the beginning and end of the match for the first
     subexpression, and so on.  In general, element number 2N
     corresponds to `(match-beginning N)'; and element number 2N + 1
     corresponds to `(match-end N)'.

     Normally all the elements are markers or `nil', but if INTEGERS is
     non-`nil', that means to use integers instead of markers.  (In
     that case, the buffer itself is appended as an additional element
     at the end of the list, to facilitate complete restoration of the
     match data.)  If the last match was done on a string with
     `string-match', then integers are always used, since markers can't
     point into a string.

     If REUSE is non-`nil', it should be a list.  In that case,
     `match-data' stores the match data in REUSE.  That is, REUSE is
     destructively modified.  REUSE does not need to have the right
     length.  If it is not long enough to contain the match data, it is
     extended.  If it is too long, the length of REUSE stays the same,
     but the elements that were not used are set to `nil'.  The purpose
     of this feature is to reduce the need for garbage collection.

     If RESEAT is non-`nil', all markers on the REUSE list are reseated
     to point to nowhere.

     As always, there must be no possibility of intervening searches
     between the call to a search function and the call to `match-data'
     that is intended to access the match data for that search.

          (match-data)
               =>  (#<marker at 9 in foo>
                    #<marker at 17 in foo>
                    #<marker at 13 in foo>
                    #<marker at 17 in foo>)

 -- Function: set-match-data match-list &optional reseat
     This function sets the match data from the elements of MATCH-LIST,
     which should be a list that was the value of a previous call to
     `match-data'.  (More precisely, anything that has the same format
     will work.)

     If MATCH-LIST refers to a buffer that doesn't exist, you don't get
     an error; that sets the match data in a meaningless but harmless
     way.

     If RESEAT is non-`nil', all markers on the MATCH-LIST list are
     reseated to point to nowhere.

     `store-match-data' is a semi-obsolete alias for `set-match-data'.


File: elisp,  Node: Saving Match Data,  Prev: Entire Match Data,  Up: Match Data

34.6.4 Saving and Restoring the Match Data
------------------------------------------

When you call a function that may search, you may need to save and
restore the match data around that call, if you want to preserve the
match data from an earlier search for later use.  Here is an example
that shows the problem that arises if you fail to save the match data:

     (re-search-forward "The \\(cat \\)")
          => 48
     (foo)                   ; `foo' does more searching.
     (match-end 0)
          => 61              ; Unexpected result--not 48!

   You can save and restore the match data with `save-match-data':

 -- Macro: save-match-data body...
     This macro executes BODY, saving and restoring the match data
     around it.  The return value is the value of the last form in BODY.

   You could use `set-match-data' together with `match-data' to imitate
the effect of the special form `save-match-data'.  Here is how:

     (let ((data (match-data)))
       (unwind-protect
           ...   ; Ok to change the original match data.
         (set-match-data data)))

   Emacs automatically saves and restores the match data when it runs
process filter functions (*note Filter Functions::) and process
sentinels (*note Sentinels::).


File: elisp,  Node: Search and Replace,  Next: Standard Regexps,  Prev: Match Data,  Up: Searching and Matching

34.7 Search and Replace
=======================

If you want to find all matches for a regexp in part of the buffer, and
replace them, the best way is to write an explicit loop using
`re-search-forward' and `replace-match', like this:

     (while (re-search-forward "foo[ \t]+bar" nil t)
       (replace-match "foobar"))

*Note Replacing the Text that Matched: Replacing Match, for a
description of `replace-match'.

   However, replacing matches in a string is more complex, especially
if you want to do it efficiently.  So Emacs provides a function to do
this.

 -- Function: replace-regexp-in-string regexp rep string &optional
          fixedcase literal subexp start
     This function copies STRING and searches it for matches for
     REGEXP, and replaces them with REP.  It returns the modified copy.
     If START is non-`nil', the search for matches starts at that index
     in STRING, so matches starting before that index are not changed.

     This function uses `replace-match' to do the replacement, and it
     passes the optional arguments FIXEDCASE, LITERAL and SUBEXP along
     to `replace-match'.

     Instead of a string, REP can be a function.  In that case,
     `replace-regexp-in-string' calls REP for each match, passing the
     text of the match as its sole argument.  It collects the value REP
     returns and passes that to `replace-match' as the replacement
     string.  The match data at this point are the result of matching
     REGEXP against a substring of STRING.

   If you want to write a command along the lines of `query-replace',
you can use `perform-replace' to do the work.

 -- Function: perform-replace from-string replacements query-flag
          regexp-flag delimited-flag &optional repeat-count map start
          end
     This function is the guts of `query-replace' and related commands.
     It searches for occurrences of FROM-STRING in the text between
     positions START and END and replaces some or all of them.  If
     START is `nil' (or omitted), point is used instead, and the end of
     the buffer's accessible portion is used for END.

     If QUERY-FLAG is `nil', it replaces all occurrences; otherwise, it
     asks the user what to do about each one.

     If REGEXP-FLAG is non-`nil', then FROM-STRING is considered a
     regular expression; otherwise, it must match literally.  If
     DELIMITED-FLAG is non-`nil', then only replacements surrounded by
     word boundaries are considered.

     The argument REPLACEMENTS specifies what to replace occurrences
     with.  If it is a string, that string is used.  It can also be a
     list of strings, to be used in cyclic order.

     If REPLACEMENTS is a cons cell, `(FUNCTION . DATA)', this means to
     call FUNCTION after each match to get the replacement text.  This
     function is called with two arguments: DATA, and the number of
     replacements already made.

     If REPEAT-COUNT is non-`nil', it should be an integer.  Then it
     specifies how many times to use each of the strings in the
     REPLACEMENTS list before advancing cyclically to the next one.

     If FROM-STRING contains upper-case letters, then `perform-replace'
     binds `case-fold-search' to `nil', and it uses the REPLACEMENTS
     without altering their case.

     Normally, the keymap `query-replace-map' defines the possible user
     responses for queries.  The argument MAP, if non-`nil', specifies
     a keymap to use instead of `query-replace-map'.

     This function uses one of two functions to search for the next
     occurrence of FROM-STRING.  These functions are specified by the
     values of two variables: `replace-re-search-function' and
     `replace-search-function'.  The former is called when the argument
     REGEXP-FLAG is non-`nil', the latter when it is `nil'.

 -- Variable: query-replace-map
     This variable holds a special keymap that defines the valid user
     responses for `perform-replace' and the commands that use it, as
     well as `y-or-n-p' and `map-y-or-n-p'.  This map is unusual in two
     ways:

        * The "key bindings" are not commands, just symbols that are
          meaningful to the functions that use this map.

        * Prefix keys are not supported; each key binding must be for a
          single-event key sequence.  This is because the functions
          don't use `read-key-sequence' to get the input; instead, they
          read a single event and look it up "by hand".

   Here are the meaningful "bindings" for `query-replace-map'.  Several
of them are meaningful only for `query-replace' and friends.

`act'
     Do take the action being considered--in other words, "yes".

`skip'
     Do not take action for this question--in other words, "no".

`exit'
     Answer this question "no", and give up on the entire series of
     questions, assuming that the answers will be "no".

`exit-prefix'
     Like `exit', but add the key that was pressed to
     `unread-comment-events'.

`act-and-exit'
     Answer this question "yes", and give up on the entire series of
     questions, assuming that subsequent answers will be "no".

`act-and-show'
     Answer this question "yes", but show the results--don't advance yet
     to the next question.

`automatic'
     Answer this question and all subsequent questions in the series
     with "yes", without further user interaction.

`backup'
     Move back to the previous place that a question was asked about.

`edit'
     Enter a recursive edit to deal with this question--instead of any
     other action that would normally be taken.

`edit-replacement'
     Edit the replacement for this question in the minibuffer.

`delete-and-edit'
     Delete the text being considered, then enter a recursive edit to
     replace it.

`recenter'
     Redisplay and center the window, then ask the same question again.

`quit'
     Perform a quit right away.  Only `y-or-n-p' and related functions
     use this answer.

`help'
     Display some help, then ask again.

 -- Variable: multi-query-replace-map
     This variable holds a keymap that extends `query-replace-map' by
     providing additional keybindings that are useful in multi-buffer
     replacements.  The additional "bindings" are:

    `automatic-all'
          Answer this question and all subsequent questions in the
          series with "yes", without further user interaction, for all
          remaining buffers.

    `exit-current'
          Answer this question "no", and give up on the entire series of
          questions for the current buffer.  Continue to the next
          buffer in the sequence.

 -- Variable: replace-search-function
     This variable specifies a function that `perform-replace' calls to
     search for the next string to replace.  Its default value is
     `search-forward'.  Any other value should name a function of 3
     arguments: the first 3 arguments of `search-forward' (*note String
     Search::).

 -- Variable: replace-re-search-function
     This variable specifies a function that `perform-replace' calls to
     search for the next regexp to replace.  Its default value is
     `re-search-forward'.  Any other value should name a function of 3
     arguments: the first 3 arguments of `re-search-forward' (*note
     Regexp Search::).


File: elisp,  Node: Standard Regexps,  Prev: Search and Replace,  Up: Searching and Matching

34.8 Standard Regular Expressions Used in Editing
=================================================

This section describes some variables that hold regular expressions
used for certain purposes in editing:

 -- User Option: page-delimiter
     This is the regular expression describing line-beginnings that
     separate pages.  The default value is `"^\014"' (i.e., `"^^L"' or
     `"^\C-l"'); this matches a line that starts with a formfeed
     character.

   The following two regular expressions should _not_ assume the match
always starts at the beginning of a line; they should not use `^' to
anchor the match.  Most often, the paragraph commands do check for a
match only at the beginning of a line, which means that `^' would be
superfluous.  When there is a nonzero left margin, they accept matches
that start after the left margin.  In that case, a `^' would be
incorrect.  However, a `^' is harmless in modes where a left margin is
never used.

 -- User Option: paragraph-separate
     This is the regular expression for recognizing the beginning of a
     line that separates paragraphs.  (If you change this, you may have
     to change `paragraph-start' also.)  The default value is
     `"[ \t\f]*$"', which matches a line that consists entirely of
     spaces, tabs, and form feeds (after its left margin).

 -- User Option: paragraph-start
     This is the regular expression for recognizing the beginning of a
     line that starts _or_ separates paragraphs.  The default value is
     `"\f\\|[ \t]*$"', which matches a line containing only whitespace
     or starting with a form feed (after its left margin).

 -- User Option: sentence-end
     If non-`nil', the value should be a regular expression describing
     the end of a sentence, including the whitespace following the
     sentence.  (All paragraph boundaries also end sentences,
     regardless.)

     If the value is `nil', as it is by default, then the function
     `sentence-end' constructs the regexp.  That is why you should
     always call the function `sentence-end' to obtain the regexp to be
     used to recognize the end of a sentence.

 -- Function: sentence-end
     This function returns the value of the variable `sentence-end', if
     non-`nil'.  Otherwise it returns a default value based on the
     values of the variables `sentence-end-double-space' (*note
     Definition of sentence-end-double-space::),
     `sentence-end-without-period', and `sentence-end-without-space'.


File: elisp,  Node: Syntax Tables,  Next: Abbrevs,  Prev: Searching and Matching,  Up: Top

35 Syntax Tables
****************

A "syntax table" specifies the syntactic role of each character in a
buffer.  It can be used to determine where words, symbols, and other
syntactic constructs begin and end.  This information is used by many
Emacs facilities, including Font Lock mode (*note Font Lock Mode::) and
the various complex movement commands (*note Motion::).

* Menu:

* Basics: Syntax Basics.     Basic concepts of syntax tables.
* Syntax Descriptors::       How characters are classified.
* Syntax Table Functions::   How to create, examine and alter syntax tables.
* Syntax Properties::        Overriding syntax with text properties.
* Motion and Syntax::        Moving over characters with certain syntaxes.
* Parsing Expressions::      Parsing balanced expressions
                                using the syntax table.
* Standard Syntax Tables::   Syntax tables used by various major modes.
* Syntax Table Internals::   How syntax table information is stored.
* Categories::               Another way of classifying character syntax.


File: elisp,  Node: Syntax Basics,  Next: Syntax Descriptors,  Up: Syntax Tables

35.1 Syntax Table Concepts
==========================

A syntax table is a char-table (*note Char-Tables::).  The element at
index C describes the character with code C.  The element's value
should be a list that encodes the syntax of the character in question.

   Syntax tables are used only for moving across text, not for the Emacs
Lisp reader.  Emacs Lisp uses built-in syntactic rules when reading Lisp
expressions, and these rules cannot be changed.  (Some Lisp systems
provide ways to redefine the read syntax, but we decided to leave this
feature out of Emacs Lisp for simplicity.)

   Each buffer has its own major mode, and each major mode has its own
idea of the syntactic class of various characters.  For example, in
Lisp mode, the character `;' begins a comment, but in C mode, it
terminates a statement.  To support these variations, Emacs makes the
syntax table local to each buffer.  Typically, each major mode has its
own syntax table and installs that table in each buffer that uses that
mode.  Changing this table alters the syntax in all those buffers as
well as in any buffers subsequently put in that mode.  Occasionally
several similar modes share one syntax table.  *Note Example Major
Modes::, for an example of how to set up a syntax table.

   A syntax table can inherit the data for some characters from the
standard syntax table, while specifying other characters itself.  The
"inherit" syntax class means "inherit this character's syntax from the
standard syntax table".  Just changing the standard syntax for a
character affects all syntax tables that inherit from it.

 -- Function: syntax-table-p object
     This function returns `t' if OBJECT is a syntax table.


File: elisp,  Node: Syntax Descriptors,  Next: Syntax Table Functions,  Prev: Syntax Basics,  Up: Syntax Tables

35.2 Syntax Descriptors
=======================

The syntactic role of a character is called its "syntax class".  Each
syntax table specifies the syntax class of each character.  There is no
necessary relationship between the class of a character in one syntax
table and its class in any other table.

   Each syntax class is designated by a mnemonic character, which
serves as the name of the class when you need to specify a class.
Usually, this designator character is one that is often assigned that
class; however, its meaning as a designator is unvarying and
independent of what syntax that character currently has.  Thus, `\' as
a designator character always means "escape character" syntax,
regardless of whether the `\' character actually has that syntax in the
current syntax table.  *Note Syntax Class Table::, for a list of syntax
classes.

   A "syntax descriptor" is a Lisp string that describes the syntax
classes and other syntactic properties of a character.  When you want
to modify the syntax of a character, that is done by calling the
function `modify-syntax-entry' and passing a syntax descriptor as one
of its arguments (*note Syntax Table Functions::).

   The first character in a syntax descriptor designates the syntax
class.  The second character specifies a matching character (e.g. in
Lisp, the matching character for `(' is `)'); if there is no matching
character, put a space there.  Then come the characters for any desired
flags.

   If no matching character or flags are needed, only one character
(specifying the syntax class) is sufficient.

   For example, the syntax descriptor for the character `*' in C mode
is `". 23"' (i.e., punctuation, matching character slot unused, second
character of a comment-starter, first character of a comment-ender),
and the entry for `/' is `. 14' (i.e., punctuation, matching character
slot unused, first character of a comment-starter, second character of
a comment-ender).

* Menu:

* Syntax Class Table::      Table of syntax classes.
* Syntax Flags::            Additional flags each character can have.


File: elisp,  Node: Syntax Class Table,  Next: Syntax Flags,  Up: Syntax Descriptors

35.2.1 Table of Syntax Classes
------------------------------

Here is a table of syntax classes, the characters that designate them,
their meanings, and examples of their use.

Whitespace characters: ` ' or `-'
     Characters that separate symbols and words from each other.
     Typically, whitespace characters have no other syntactic
     significance, and multiple whitespace characters are syntactically
     equivalent to a single one.  Space, tab, and formfeed are
     classified as whitespace in almost all major modes.

     This syntax class can be designated by either ` ' or `-'.  Both
     designators are equivalent.

Word constituents: `w'
     Parts of words in human languages.  These are typically used in
     variable and command names in programs.  All upper- and lower-case
     letters, and the digits, are typically word constituents.

Symbol constituents: `_'
     Extra characters used in variable and command names along with word
     constituents.  Examples include the characters `$&*+-_<>' in Lisp
     mode, which may be part of a symbol name even though they are not
     part of English words.  In standard C, the only
     non-word-constituent character that is valid in symbols is
     underscore (`_').

Punctuation characters: `.'
     Characters used as punctuation in a human language, or used in a
     programming language to separate symbols from one another.  Some
     programming language modes, such as Emacs Lisp mode, have no
     characters in this class since the few characters that are not
     symbol or word constituents all have other uses.  Other
     programming language modes, such as C mode, use punctuation syntax
     for operators.

Open parenthesis characters: `('
Close parenthesis characters: `)'
     Characters used in dissimilar pairs to surround sentences or
     expressions.  Such a grouping is begun with an open parenthesis
     character and terminated with a close.  Each open parenthesis
     character matches a particular close parenthesis character, and
     vice versa.  Normally, Emacs indicates momentarily the matching
     open parenthesis when you insert a close parenthesis.  *Note
     Blinking::.

     In human languages, and in C code, the parenthesis pairs are `()',
     `[]', and `{}'.  In Emacs Lisp, the delimiters for lists and
     vectors (`()' and `[]') are classified as parenthesis characters.

String quotes: `"'
     Characters used to delimit string constants.  The same string quote
     character appears at the beginning and the end of a string.  Such
     quoted strings do not nest.

     The parsing facilities of Emacs consider a string as a single
     token.  The usual syntactic meanings of the characters in the
     string are suppressed.

     The Lisp modes have two string quote characters: double-quote (`"')
     and vertical bar (`|').  `|' is not used in Emacs Lisp, but it is
     used in Common Lisp.  C also has two string quote characters:
     double-quote for strings, and single-quote (`'') for character
     constants.

     Human text has no string quote characters.  We do not want
     quotation marks to turn off the usual syntactic properties of
     other characters in the quotation.

Escape-syntax characters: `\'
     Characters that start an escape sequence, such as is used in string
     and character constants.  The character `\' belongs to this class
     in both C and Lisp.  (In C, it is used thus only inside strings,
     but it turns out to cause no trouble to treat it this way
     throughout C code.)

     Characters in this class count as part of words if
     `words-include-escapes' is non-`nil'.  *Note Word Motion::.

Character quotes: `/'
     Characters used to quote the following character so that it loses
     its normal syntactic meaning.  This differs from an escape
     character in that only the character immediately following is ever
     affected.

     Characters in this class count as part of words if
     `words-include-escapes' is non-`nil'.  *Note Word Motion::.

     This class is used for backslash in TeX mode.

Paired delimiters: `$'
     Similar to string quote characters, except that the syntactic
     properties of the characters between the delimiters are not
     suppressed.  Only TeX mode uses a paired delimiter presently--the
     `$' that both enters and leaves math mode.

Expression prefixes: `''
     Characters used for syntactic operators that are considered as
     part of an expression if they appear next to one.  In Lisp modes,
     these characters include the apostrophe, `'' (used for quoting),
     the comma, `,' (used in macros), and `#' (used in the read syntax
     for certain data types).

Comment starters: `<'
Comment enders: `>'
     Characters used in various languages to delimit comments.  Human
     text has no comment characters.  In Lisp, the semicolon (`;')
     starts a comment and a newline or formfeed ends one.

Inherit standard syntax: `@'
     This syntax class does not specify a particular syntax.  It says to
     look in the standard syntax table to find the syntax of this
     character.

Generic comment delimiters: `!'
     Characters that start or end a special kind of comment.  _Any_
     generic comment delimiter matches _any_ generic comment delimiter,
     but they cannot match a comment starter or comment ender; generic
     comment delimiters can only match each other.

     This syntax class is primarily meant for use with the
     `syntax-table' text property (*note Syntax Properties::).  You can
     mark any range of characters as forming a comment, by giving the
     first and last characters of the range `syntax-table' properties
     identifying them as generic comment delimiters.

Generic string delimiters: `|'
     Characters that start or end a string.  This class differs from the
     string quote class in that _any_ generic string delimiter can
     match any other generic string delimiter; but they do not match
     ordinary string quote characters.

     This syntax class is primarily meant for use with the
     `syntax-table' text property (*note Syntax Properties::).  You can
     mark any range of characters as forming a string constant, by
     giving the first and last characters of the range `syntax-table'
     properties identifying them as generic string delimiters.


File: elisp,  Node: Syntax Flags,  Prev: Syntax Class Table,  Up: Syntax Descriptors

35.2.2 Syntax Flags
-------------------

In addition to the classes, entries for characters in a syntax table
can specify flags.  There are eight possible flags, represented by the
characters `1', `2', `3', `4', `b', `c', `n', and `p'.

   All the flags except `p' are used to describe comment delimiters.
The digit flags are used for comment delimiters made up of 2
characters.  They indicate that a character can _also_ be part of a
comment sequence, in addition to the syntactic properties associated
with its character class.  The flags are independent of the class and
each other for the sake of characters such as `*' in C mode, which is a
punctuation character, _and_ the second character of a start-of-comment
sequence (`/*'), _and_ the first character of an end-of-comment
sequence (`*/').  The flags `b', `c', and `n' are used to qualify the
corresponding comment delimiter.

   Here is a table of the possible flags for a character C, and what
they mean:

   * `1' means C is the start of a two-character comment-start sequence.

   * `2' means C is the second character of such a sequence.

   * `3' means C is the start of a two-character comment-end sequence.

   * `4' means C is the second character of such a sequence.

   * `b' means that C as a comment delimiter belongs to the alternative
     "b" comment style.  For a two-character comment starter, this flag
     is only significant on the second char, and for a 2-character
     comment ender it is only significant on the first char.

   * `c' means that C as a comment delimiter belongs to the alternative
     "c" comment style.  For a two-character comment delimiter, `c' on
     either character makes it of style "c".

   * `n' on a comment delimiter character specifies that this kind of
     comment can be nested.  For a two-character comment delimiter, `n'
     on either character makes it nestable.

     Emacs supports several comment styles simultaneously in any one
     syntax table.  A comment style is a set of flags `b', `c', and
     `n', so there can be up to 8 different comment styles.  Each
     comment delimiter has a style and only matches comment delimiters
     of the same style.  Thus if a comment starts with the comment-start
     sequence of style "bn", it will extend until the next matching
     comment-end sequence of style "bn".

     The appropriate comment syntax settings for C++ can be as follows:

    `/'
          `124'

    `*'
          `23b'

    newline
          `>'

     This defines four comment-delimiting sequences:

    `/*'
          This is a comment-start sequence for "b" style because the
          second character, `*', has the `b' flag.

    `//'
          This is a comment-start sequence for "a" style because the
          second character, `/', does not have the `b' flag.

    `*/'
          This is a comment-end sequence for "b" style because the first
          character, `*', has the `b' flag.

    newline
          This is a comment-end sequence for "a" style, because the
          newline character does not have the `b' flag.

   * `p' identifies an additional "prefix character" for Lisp syntax.
     These characters are treated as whitespace when they appear between
     expressions.  When they appear within an expression, they are
     handled according to their usual syntax classes.

     The function `backward-prefix-chars' moves back over these
     characters, as well as over characters whose primary syntax class
     is prefix (`'').  *Note Motion and Syntax::.


File: elisp,  Node: Syntax Table Functions,  Next: Syntax Properties,  Prev: Syntax Descriptors,  Up: Syntax Tables

35.3 Syntax Table Functions
===========================

In this section we describe functions for creating, accessing and
altering syntax tables.

 -- Function: make-syntax-table &optional table
     This function creates a new syntax table, with all values
     initialized to `nil'.  If TABLE is non-`nil', it becomes the
     parent of the new syntax table, otherwise the standard syntax
     table is the parent.  Like all char-tables, a syntax table
     inherits from its parent.  Thus the original syntax of all
     characters in the returned syntax table is determined by the
     parent.  *Note Char-Tables::.

     Most major mode syntax tables are created in this way.

 -- Function: copy-syntax-table &optional table
     This function constructs a copy of TABLE and returns it.  If TABLE
     is not supplied (or is `nil'), it returns a copy of the standard
     syntax table.  Otherwise, an error is signaled if TABLE is not a
     syntax table.

 -- Command: modify-syntax-entry char syntax-descriptor &optional table
     This function sets the syntax entry for CHAR according to
     SYNTAX-DESCRIPTOR.  CHAR must be a character, or a cons cell of
     the form `(MIN . MAX)'; in the latter case, the function sets the
     syntax entries for all characters in the range between MIN and
     MAX, inclusive.

     The syntax is changed only for TABLE, which defaults to the
     current buffer's syntax table, and not in any other syntax table.

     The argument SYNTAX-DESCRIPTOR is a syntax descriptor for the
     desired syntax (i.e. a string beginning with a class designator
     character, and optionally containing a matching character and
     syntax flags).  An error is signaled if the first character is not
     one of the seventeen syntax class designators.  *Note Syntax
     Descriptors::.

     This function always returns `nil'.  The old syntax information in
     the table for this character is discarded.

     Examples:

          ;; Put the space character in class whitespace.
          (modify-syntax-entry ?\s " ")
               => nil

          ;; Make `$' an open parenthesis character,
          ;;   with `^' as its matching close.
          (modify-syntax-entry ?$ "(^")
               => nil

          ;; Make `^' a close parenthesis character,
          ;;   with `$' as its matching open.
          (modify-syntax-entry ?^ ")$")
               => nil

          ;; Make `/' a punctuation character,
          ;;   the first character of a start-comment sequence,
          ;;   and the second character of an end-comment sequence.
          ;;   This is used in C mode.
          (modify-syntax-entry ?/ ". 14")
               => nil

 -- Function: char-syntax character
     This function returns the syntax class of CHARACTER, represented
     by its mnemonic designator character.  This returns _only_ the
     class, not any matching parenthesis or flags.

     An error is signaled if CHAR is not a character.

     The following examples apply to C mode.  The first example shows
     that the syntax class of space is whitespace (represented by a
     space).  The second example shows that the syntax of `/' is
     punctuation.  This does not show the fact that it is also part of
     comment-start and -end sequences.  The third example shows that
     open parenthesis is in the class of open parentheses.  This does
     not show the fact that it has a matching character, `)'.

          (string (char-syntax ?\s))
               => " "

          (string (char-syntax ?/))
               => "."

          (string (char-syntax ?\())
               => "("

     We use `string' to make it easier to see the character returned by
     `char-syntax'.

 -- Function: set-syntax-table table
     This function makes TABLE the syntax table for the current buffer.
     It returns TABLE.

 -- Function: syntax-table
     This function returns the current syntax table, which is the table
     for the current buffer.

 -- Macro: with-syntax-table TABLE BODY...
     This macro executes BODY using TABLE as the current syntax table.
     It returns the value of the last form in BODY, after restoring the
     old current syntax table.

     Since each buffer has its own current syntax table, we should make
     that more precise: `with-syntax-table' temporarily alters the
     current syntax table of whichever buffer is current at the time
     the macro execution starts.  Other buffers are not affected.


File: elisp,  Node: Syntax Properties,  Next: Motion and Syntax,  Prev: Syntax Table Functions,  Up: Syntax Tables

35.4 Syntax Properties
======================

When the syntax table is not flexible enough to specify the syntax of a
language, you can override the syntax table for specific character
occurrences in the buffer, by applying a `syntax-table' text property.
*Note Text Properties::, for how to apply text properties.

   The valid values of `syntax-table' text property are:

SYNTAX-TABLE
     If the property value is a syntax table, that table is used
     instead of the current buffer's syntax table to determine the
     syntax for the underlying text character.

`(SYNTAX-CODE . MATCHING-CHAR)'
     A cons cell of this format specifies the syntax for the underlying
     text character.  (*note Syntax Table Internals::)

`nil'
     If the property is `nil', the character's syntax is determined from
     the current syntax table in the usual way.

 -- Variable: parse-sexp-lookup-properties
     If this is non-`nil', the syntax scanning functions, like
     `forward-sexp', pay attention to syntax text properties.
     Otherwise they use only the current syntax table.

 -- Variable: syntax-propertize-function
     This variable, if non-`nil', should store a function for applying
     `syntax-table' properties to a specified stretch of text.  It is
     intended to be used by major modes to install a function which
     applies `syntax-table' properties in some mode-appropriate way.

     The function is called by `syntax-ppss' (*note Position Parse::),
     and by Font Lock mode during syntactic fontification (*note
     Syntactic Font Lock::).  It is called with two arguments, START and
     END, which are the starting and ending positions of the text on
     which it should act.  It is allowed to call `syntax-ppss' on any
     position before END.  However, it should not call
     `syntax-ppss-flush-cache'; so, it is not allowed to call
     `syntax-ppss' on some position and later modify the buffer at an
     earlier position.

 -- Variable: syntax-propertize-extend-region-functions
     This abnormal hook is run by the syntax parsing code prior to
     calling `syntax-propertize-function'.  Its role is to help locate
     safe starting and ending buffer positions for passing to
     `syntax-propertize-function'.  For example, a major mode can add a
     function to this hook to identify multi-line syntactic constructs,
     and ensure that the boundaries do not fall in the middle of one.

     Each function in this hook should accept two arguments, START and
     END.  It should return either a cons cell of two adjusted buffer
     positions, `(NEW-START . NEW-END)', or `nil' if no adjustment is
     necessary.  The hook functions are run in turn, repeatedly, until
     they all return `nil'.


File: elisp,  Node: Motion and Syntax,  Next: Parsing Expressions,  Prev: Syntax Properties,  Up: Syntax Tables

35.5 Motion and Syntax
======================

This section describes functions for moving across characters that have
certain syntax classes.

 -- Function: skip-syntax-forward syntaxes &optional limit
     This function moves point forward across characters having syntax
     classes mentioned in SYNTAXES (a string of syntax class
     characters).  It stops when it encounters the end of the buffer, or
     position LIMIT (if specified), or a character it is not supposed
     to skip.

     If SYNTAXES starts with `^', then the function skips characters
     whose syntax is _not_ in SYNTAXES.

     The return value is the distance traveled, which is a nonnegative
     integer.

 -- Function: skip-syntax-backward syntaxes &optional limit
     This function moves point backward across characters whose syntax
     classes are mentioned in SYNTAXES.  It stops when it encounters
     the beginning of the buffer, or position LIMIT (if specified), or
     a character it is not supposed to skip.

     If SYNTAXES starts with `^', then the function skips characters
     whose syntax is _not_ in SYNTAXES.

     The return value indicates the distance traveled.  It is an
     integer that is zero or less.

 -- Function: backward-prefix-chars
     This function moves point backward over any number of characters
     with expression prefix syntax.  This includes both characters in
     the expression prefix syntax class, and characters with the `p'
     flag.


File: elisp,  Node: Parsing Expressions,  Next: Standard Syntax Tables,  Prev: Motion and Syntax,  Up: Syntax Tables

35.6 Parsing Expressions
========================

This section describes functions for parsing and scanning balanced
expressions.  We will refer to such expressions as "sexps", following
the terminology of Lisp, even though these functions can act on
languages other than Lisp.  Basically, a sexp is either a balanced
parenthetical grouping, a string, or a "symbol" (i.e. a sequence of
characters whose syntax is either word constituent or symbol
constituent).  However, characters in the expression prefix syntax
class (*note Syntax Class Table::) are treated as part of the sexp if
they appear next to it.

   The syntax table controls the interpretation of characters, so these
functions can be used for Lisp expressions when in Lisp mode and for C
expressions when in C mode.  *Note List Motion::, for convenient
higher-level functions for moving over balanced expressions.

   A character's syntax controls how it changes the state of the
parser, rather than describing the state itself.  For example, a string
delimiter character toggles the parser state between "in-string" and
"in-code", but the syntax of characters does not directly say whether
they are inside a string.  For example (note that 15 is the syntax code
for generic string delimiters),

     (put-text-property 1 9 'syntax-table '(15 . nil))

does not tell Emacs that the first eight chars of the current buffer
are a string, but rather that they are all string delimiters.  As a
result, Emacs treats them as four consecutive empty string constants.

* Menu:

* Motion via Parsing::       Motion functions that work by parsing.
* Position Parse::           Determining the syntactic state of a position.
* Parser State::             How Emacs represents a syntactic state.
* Low-Level Parsing::        Parsing across a specified region.
* Control Parsing::          Parameters that affect parsing.


File: elisp,  Node: Motion via Parsing,  Next: Position Parse,  Up: Parsing Expressions

35.6.1 Motion Commands Based on Parsing
---------------------------------------

This section describes simple point-motion functions that operate based
on parsing expressions.

 -- Function: scan-lists from count depth
     This function scans forward COUNT balanced parenthetical groupings
     from position FROM.  It returns the position where the scan stops.
     If COUNT is negative, the scan moves backwards.

     If DEPTH is nonzero, treat the starting position as being DEPTH
     parentheses deep.  The scanner moves forward or backward through
     the buffer until the depth changes to zero COUNT times.  Hence, a
     positive value for DEPTH has the effect of moving out DEPTH levels
     of parenthesis from the starting position, while a negative DEPTH
     has the effect of moving deeper by -DEPTH levels of parenthesis.

     Scanning ignores comments if `parse-sexp-ignore-comments' is
     non-`nil'.

     If the scan reaches the beginning or end of the accessible part of
     the buffer before it has scanned over COUNT parenthetical
     groupings, the return value is `nil' if the depth at that point is
     zero; if the depth is non-zero, a `scan-error' error is signaled.

 -- Function: scan-sexps from count
     This function scans forward COUNT sexps from position FROM.  It
     returns the position where the scan stops.  If COUNT is negative,
     the scan moves backwards.

     Scanning ignores comments if `parse-sexp-ignore-comments' is
     non-`nil'.

     If the scan reaches the beginning or end of (the accessible part
     of) the buffer while in the middle of a parenthetical grouping, an
     error is signaled.  If it reaches the beginning or end between
     groupings but before count is used up, `nil' is returned.

 -- Function: forward-comment count
     This function moves point forward across COUNT complete comments
       (that is, including the starting delimiter and the terminating
     delimiter if any), plus any whitespace encountered on the way.  It
     moves backward if COUNT is negative.  If it encounters anything
     other than a comment or whitespace, it stops, leaving point at the
     place where it stopped.  This includes (for instance) finding the
     end of a comment when moving forward and expecting the beginning
     of one.  The function also stops immediately after moving over the
     specified number of complete comments.  If COUNT comments are
     found as expected, with nothing except whitespace between them, it
     returns `t'; otherwise it returns `nil'.

     This function cannot tell whether the "comments" it traverses are
     embedded within a string.  If they look like comments, it treats
     them as comments.

     To move forward over all comments and whitespace following point,
     use `(forward-comment (buffer-size))'.  `(buffer-size)' is a good
     argument to use, because the number of comments in the buffer
     cannot exceed that many.


File: elisp,  Node: Position Parse,  Next: Parser State,  Prev: Motion via Parsing,  Up: Parsing Expressions

35.6.2 Finding the Parse State for a Position
---------------------------------------------

For syntactic analysis, such as in indentation, often the useful thing
is to compute the syntactic state corresponding to a given buffer
position.  This function does that conveniently.

 -- Function: syntax-ppss &optional pos
     This function returns the parser state that the parser would reach
     at position POS starting from the beginning of the buffer.  *Note
     Parser State::, for a description of the parser state.

     The return value is the same as if you call the low-level parsing
     function `parse-partial-sexp' to parse from the beginning of the
     buffer to POS (*note Low-Level Parsing::).  However, `syntax-ppss'
     uses a cache to speed up the computation.  Due to this
     optimization, the second value (previous complete subexpression)
     and sixth value (minimum parenthesis depth) in the returned parser
     state are not meaningful.

     This function has a side effect: it adds a buffer-local entry to
     `before-change-functions' (*note Change Hooks::) for
     `syntax-ppss-flush-cache' (see below).  This entry keeps the cache
     consistent as the buffer is modified.  However, the cache might
     not be updated if `syntax-ppss' is called while
     `before-change-functions' is temporarily let-bound, or if the
     buffer is modified without running the hook, such as when using
     `inhibit-modification-hooks'.  In those cases, it is necessary to
     call `syntax-ppss-flush-cache' explicitly.

 -- Function: syntax-ppss-flush-cache beg &rest ignored-args
     This function flushes the cache used by `syntax-ppss', starting at
     position BEG.  The remaining arguments, IGNORED-ARGS, are ignored;
     this function accepts them so that it can be directly used on
     hooks such as `before-change-functions' (*note Change Hooks::).

   Major modes can make `syntax-ppss' run faster by specifying where it
needs to start parsing.

 -- Variable: syntax-begin-function
     If this is non-`nil', it should be a function that moves to an
     earlier buffer position where the parser state is equivalent to
     `nil'--in other words, a position outside of any comment, string,
     or parenthesis.  `syntax-ppss' uses it to further optimize its
     computations, when the cache gives no help.


File: elisp,  Node: Parser State,  Next: Low-Level Parsing,  Prev: Position Parse,  Up: Parsing Expressions

35.6.3 Parser State
-------------------

A "parser state" is a list of ten elements describing the state of the
syntactic parser, after it parses the text between a specified starting
point and a specified end point in the buffer.  Parsing functions such
as `syntax-ppss' (*note Position Parse::) return a parser state as the
value.  Some parsing functions accept a parser state as an argument,
for resuming parsing.

   Here are the meanings of the elements of the parser state:

  0. The depth in parentheses, counting from 0.  *Warning:* this can be
     negative if there are more close parens than open parens between
     the parser's starting point and end point.

  1. The character position of the start of the innermost parenthetical
     grouping containing the stopping point; `nil' if none.

  2. The character position of the start of the last complete
     subexpression terminated; `nil' if none.

  3. Non-`nil' if inside a string.  More precisely, this is the
     character that will terminate the string, or `t' if a generic
     string delimiter character should terminate it.

  4. `t' if inside a non-nestable comment (of any comment style; *note
     Syntax Flags::); or the comment nesting level if inside a comment
     that can be nested.

  5. `t' if the end point is just after a quote character.

  6. The minimum parenthesis depth encountered during this scan.

  7. What kind of comment is active: `nil' if not in a comment or in a
     comment of style `a'; 1 for a comment of style `b'; 2 for a
     comment of style `c'; and `syntax-table' for a comment that should
     be ended by a generic comment delimiter character.

  8. The string or comment start position.  While inside a comment,
     this is the position where the comment began; while inside a
     string, this is the position where the string began.  When outside
     of strings and comments, this element is `nil'.

  9. Internal data for continuing the parsing.  The meaning of this
     data is subject to change; it is used if you pass this list as the
     STATE argument to another call.

   Elements 1, 2, and 6 are ignored in a state which you pass as an
argument to continue parsing, and elements 8 and 9 are used only in
trivial cases.  Those elements are mainly used internally by the parser
code.

   One additional piece of useful information is available from a
parser state using this function:

 -- Function: syntax-ppss-toplevel-pos state
     This function extracts, from parser state STATE, the last position
     scanned in the parse which was at top level in grammatical
     structure.  "At top level" means outside of any parentheses,
     comments, or strings.

     The value is `nil' if STATE represents a parse which has arrived
     at a top level position.


File: elisp,  Node: Low-Level Parsing,  Next: Control Parsing,  Prev: Parser State,  Up: Parsing Expressions

35.6.4 Low-Level Parsing
------------------------

The most basic way to use the expression parser is to tell it to start
at a given position with a certain state, and parse up to a specified
end position.

 -- Function: parse-partial-sexp start limit &optional target-depth
          stop-before state stop-comment
     This function parses a sexp in the current buffer starting at
     START, not scanning past LIMIT.  It stops at position LIMIT or
     when certain criteria described below are met, and sets point to
     the location where parsing stops.  It returns a parser state
     describing the status of the parse at the point where it stops.

     If the third argument TARGET-DEPTH is non-`nil', parsing stops if
     the depth in parentheses becomes equal to TARGET-DEPTH.  The depth
     starts at 0, or at whatever is given in STATE.

     If the fourth argument STOP-BEFORE is non-`nil', parsing stops
     when it comes to any character that starts a sexp.  If
     STOP-COMMENT is non-`nil', parsing stops when it comes to the
     start of a comment.  If STOP-COMMENT is the symbol `syntax-table',
     parsing stops after the start of a comment or a string, or the end
     of a comment or a string, whichever comes first.

     If STATE is `nil', START is assumed to be at the top level of
     parenthesis structure, such as the beginning of a function
     definition.  Alternatively, you might wish to resume parsing in the
     middle of the structure.  To do this, you must provide a STATE
     argument that describes the initial status of parsing.  The value
     returned by a previous call to `parse-partial-sexp' will do nicely.


File: elisp,  Node: Control Parsing,  Prev: Low-Level Parsing,  Up: Parsing Expressions

35.6.5 Parameters to Control Parsing
------------------------------------

 -- Variable: multibyte-syntax-as-symbol
     If this variable is non-`nil', `scan-sexps' treats all non-ASCII
     characters as symbol constituents regardless of what the syntax
     table says about them.  (However, text properties can still
     override the syntax.)

 -- User Option: parse-sexp-ignore-comments
     If the value is non-`nil', then comments are treated as whitespace
     by the functions in this section and by `forward-sexp',
     `scan-lists' and `scan-sexps'.

   The behavior of `parse-partial-sexp' is also affected by
`parse-sexp-lookup-properties' (*note Syntax Properties::).

   You can use `forward-comment' to move forward or backward over one
comment or several comments.


File: elisp,  Node: Standard Syntax Tables,  Next: Syntax Table Internals,  Prev: Parsing Expressions,  Up: Syntax Tables

35.7 Some Standard Syntax Tables
================================

Most of the major modes in Emacs have their own syntax tables.  Here
are several of them:

 -- Function: standard-syntax-table
     This function returns the standard syntax table, which is the
     syntax table used in Fundamental mode.

 -- Variable: text-mode-syntax-table
     The value of this variable is the syntax table used in Text mode.

 -- Variable: c-mode-syntax-table
     The value of this variable is the syntax table for C-mode buffers.

 -- Variable: emacs-lisp-mode-syntax-table
     The value of this variable is the syntax table used in Emacs Lisp
     mode by editing commands.  (It has no effect on the Lisp `read'
     function.)


File: elisp,  Node: Syntax Table Internals,  Next: Categories,  Prev: Standard Syntax Tables,  Up: Syntax Tables

35.8 Syntax Table Internals
===========================

Lisp programs don't usually work with the elements directly; the
Lisp-level syntax table functions usually work with syntax descriptors
(*note Syntax Descriptors::).  Nonetheless, here we document the
internal format.  This format is used mostly when manipulating syntax
properties.

   Each element of a syntax table is a cons cell of the form
`(SYNTAX-CODE . MATCHING-CHAR)'.  The CAR, SYNTAX-CODE, is an integer
that encodes the syntax class, and any flags.  The CDR, MATCHING-CHAR,
is non-`nil' if a character to match was specified.

   This table gives the value of SYNTAX-CODE which corresponds to each
syntactic type.

     Integer Class          Integer Class          Integer Class
     0    whitespace        5    close parenthesis 10    character quote
     1    punctuation       6    expression prefix 11    comment-start
     2    word              7    string quote      12    comment-end
     3    symbol            8    paired delimiter  13    inherit
     4    open parenthesis  9    escape            14    generic comment
     15   generic string                           

   For example, the usual syntax value for `(' is `(4 . 41)'.  (41 is
the character code for `)'.)

   The flags are encoded in higher order bits, starting 16 bits from the
least significant bit.  This table gives the power of two which
corresponds to each syntax flag.

     Prefix Flag            Prefix Flag            Prefix Flag
     `1'    `(lsh 1 16)'    `4'    `(lsh 1 19)'    `b'    `(lsh 1 21)'
     `2'    `(lsh 1 17)'    `p'    `(lsh 1 20)'    `n'    `(lsh 1 22)'
     `3'    `(lsh 1 18)'                           

 -- Function: string-to-syntax DESC
     This function returns the internal form corresponding to the syntax
     descriptor DESC, a cons cell `(SYNTAX-CODE . MATCHING-CHAR)'.

 -- Function: syntax-after pos
     This function returns the syntax code of the character in the
     buffer after position POS, taking account of syntax properties as
     well as the syntax table.  If POS is outside the buffer's
     accessible portion (*note accessible portion: Narrowing.), this
     function returns `nil'.

 -- Function: syntax-class syntax
     This function returns the syntax class of the syntax code SYNTAX.
     (It masks off the high 16 bits that hold the flags encoded in the
     syntax descriptor.)  If SYNTAX is `nil', it returns `nil'; this is
     so evaluating the expression

          (syntax-class (syntax-after pos))

     where `pos' is outside the buffer's accessible portion, will yield
     `nil' without throwing errors or producing wrong syntax class
     codes.


File: elisp,  Node: Categories,  Prev: Syntax Table Internals,  Up: Syntax Tables

35.9 Categories
===============

"Categories" provide an alternate way of classifying characters
syntactically.  You can define several categories as needed, then
independently assign each character to one or more categories.  Unlike
syntax classes, categories are not mutually exclusive; it is normal for
one character to belong to several categories.

   Each buffer has a "category table" which records which categories
are defined and also which characters belong to each category.  Each
category table defines its own categories, but normally these are
initialized by copying from the standard categories table, so that the
standard categories are available in all modes.

   Each category has a name, which is an ASCII printing character in
the range ` ' to `~'.  You specify the name of a category when you
define it with `define-category'.

   The category table is actually a char-table (*note Char-Tables::).
The element of the category table at index C is a "category set"--a
bool-vector--that indicates which categories character C belongs to.
In this category set, if the element at index CAT is `t', that means
category CAT is a member of the set, and that character C belongs to
category CAT.

   For the next three functions, the optional argument TABLE defaults
to the current buffer's category table.

 -- Function: define-category char docstring &optional table
     This function defines a new category, with name CHAR and
     documentation DOCSTRING, for the category table TABLE.

     Here's an example of defining a new category for characters that
     have strong right-to-left directionality (*note Bidirectional
     Display::) and using it in a special category table:

          (defvar special-category-table-for-bidi
            (let ((category-table (make-category-table))
          	(uniprop-table (unicode-property-table-internal 'bidi-class)))
              (define-category ?R "Characters of bidi-class R, AL, or RLO"
                               category-table)
              (map-char-table
               #'(lambda (key val)
          	 (if (memq val '(R AL RLO))
          	     (modify-category-entry key ?R category-table)))
               uniprop-table)
              category-table))

 -- Function: category-docstring category &optional table
     This function returns the documentation string of category CATEGORY
     in category table TABLE.

          (category-docstring ?a)
               => "ASCII"
          (category-docstring ?l)
               => "Latin"

 -- Function: get-unused-category &optional table
     This function returns a category name (a character) which is not
     currently defined in TABLE.  If all possible categories are in use
     in TABLE, it returns `nil'.

 -- Function: category-table
     This function returns the current buffer's category table.

 -- Function: category-table-p object
     This function returns `t' if OBJECT is a category table, otherwise
     `nil'.

 -- Function: standard-category-table
     This function returns the standard category table.

 -- Function: copy-category-table &optional table
     This function constructs a copy of TABLE and returns it.  If TABLE
     is not supplied (or is `nil'), it returns a copy of the standard
     category table.  Otherwise, an error is signaled if TABLE is not a
     category table.

 -- Function: set-category-table table
     This function makes TABLE the category table for the current
     buffer.  It returns TABLE.

 -- Function: make-category-table
     This creates and returns an empty category table.  In an empty
     category table, no categories have been allocated, and no
     characters belong to any categories.

 -- Function: make-category-set categories
     This function returns a new category set--a bool-vector--whose
     initial contents are the categories listed in the string
     CATEGORIES.  The elements of CATEGORIES should be category names;
     the new category set has `t' for each of those categories, and
     `nil' for all other categories.

          (make-category-set "al")
               => #&128"\0\0\0\0\0\0\0\0\0\0\0\0\2\20\0\0"

 -- Function: char-category-set char
     This function returns the category set for character CHAR in the
     current buffer's category table.  This is the bool-vector which
     records which categories the character CHAR belongs to.  The
     function `char-category-set' does not allocate storage, because it
     returns the same bool-vector that exists in the category table.

          (char-category-set ?a)
               => #&128"\0\0\0\0\0\0\0\0\0\0\0\0\2\20\0\0"

 -- Function: category-set-mnemonics category-set
     This function converts the category set CATEGORY-SET into a string
     containing the characters that designate the categories that are
     members of the set.

          (category-set-mnemonics (char-category-set ?a))
               => "al"

 -- Function: modify-category-entry char category &optional table reset
     This function modifies the category set of CHAR in category table
     TABLE (which defaults to the current buffer's category table).
     CHAR can be a character, or a cons cell of the form `(MIN . MAX)';
     in the latter case, the function modifies the category sets of all
     characters in the range between MIN and MAX, inclusive.

     Normally, it modifies a category set by adding CATEGORY to it.
     But if RESET is non-`nil', then it deletes CATEGORY instead.

 -- Command: describe-categories &optional buffer-or-name
     This function describes the category specifications in the current
     category table.  It inserts the descriptions in a buffer, and then
     displays that buffer.  If BUFFER-OR-NAME is non-`nil', it
     describes the category table of that buffer instead.


File: elisp,  Node: Abbrevs,  Next: Processes,  Prev: Syntax Tables,  Up: Top

36 Abbrevs and Abbrev Expansion
*******************************

An abbreviation or "abbrev" is a string of characters that may be
expanded to a longer string.  The user can insert the abbrev string and
find it replaced automatically with the expansion of the abbrev.  This
saves typing.

   The set of abbrevs currently in effect is recorded in an "abbrev
table".  Each buffer has a local abbrev table, but normally all buffers
in the same major mode share one abbrev table.  There is also a global
abbrev table.  Normally both are used.

   An abbrev table is represented as an obarray.  *Note Creating
Symbols::, for information about obarrays.  Each abbreviation is
represented by a symbol in the obarray.  The symbol's name is the
abbreviation; its value is the expansion; its function definition is
the hook function for performing the expansion (*note Defining
Abbrevs::); and its property list cell contains various additional
properties, including the use count and the number of times the
abbreviation has been expanded (*note Abbrev Properties::).

   Certain abbrevs, called "system abbrevs", are defined by a major
mode instead of the user.  A system abbrev is identified by its
non-`nil' `:system' property (*note Abbrev Properties::).  When abbrevs
are saved to an abbrev file, system abbrevs are omitted.  *Note Abbrev
Files::.

   Because the symbols used for abbrevs are not interned in the usual
obarray, they will never appear as the result of reading a Lisp
expression; in fact, normally they are never used except by the code
that handles abbrevs.  Therefore, it is safe to use them in a
nonstandard way.

   If the minor mode Abbrev mode is enabled, the buffer-local variable
`abbrev-mode' is non-`nil', and abbrevs are automatically expanded in
the buffer.  For the user-level commands for abbrevs, see *note Abbrev
Mode: (emacs)Abbrevs.

* Menu:

* Tables: Abbrev Tables.        Creating and working with abbrev tables.
* Defining Abbrevs::            Specifying abbreviations and their expansions.
* Files: Abbrev Files.          Saving abbrevs in files.
* Expansion: Abbrev Expansion.  Controlling expansion; expansion subroutines.
* Standard Abbrev Tables::      Abbrev tables used by various major modes.
* Abbrev Properties::           How to read and set abbrev properties.
                                Which properties have which effect.
* Abbrev Table Properties::     How to read and set abbrev table properties.
                                Which properties have which effect.


File: elisp,  Node: Abbrev Tables,  Next: Defining Abbrevs,  Prev: Abbrevs,  Up: Abbrevs

36.1 Abbrev Tables
==================

This section describes how to create and manipulate abbrev tables.

 -- Function: make-abbrev-table &optional props
     This function creates and returns a new, empty abbrev table--an
     obarray containing no symbols.  It is a vector filled with zeros.
     PROPS is a property list that is applied to the new table (*note
     Abbrev Table Properties::).

 -- Function: abbrev-table-p object
     This function returns a non-`nil' value if OBJECT is an abbrev
     table.

 -- Function: clear-abbrev-table abbrev-table
     This function undefines all the abbrevs in ABBREV-TABLE, leaving
     it empty.

 -- Function: copy-abbrev-table abbrev-table
     This function returns a copy of ABBREV-TABLE--a new abbrev table
     containing the same abbrev definitions.  It does _not_ copy any
     property lists; only the names, values, and functions.

 -- Function: define-abbrev-table tabname definitions &optional
          docstring &rest props
     This function defines TABNAME (a symbol) as an abbrev table name,
     i.e., as a variable whose value is an abbrev table.  It defines
     abbrevs in the table according to DEFINITIONS, a list of elements
     of the form `(ABBREVNAME EXPANSION [HOOK] [PROPS...])'.  These
     elements are passed as arguments to `define-abbrev'.   The
     optional string DOCSTRING is the documentation string of the
     variable TABNAME.  The property list PROPS is applied to the
     abbrev table (*note Abbrev Table Properties::).

     If this function is called more than once for the same TABNAME,
     subsequent calls add the definitions in DEFINITIONS to TABNAME,
     rather than overwriting the entire original contents.  (A
     subsequent call only overrides abbrevs explicitly redefined or
     undefined in DEFINITIONS.)

 -- Variable: abbrev-table-name-list
     This is a list of symbols whose values are abbrev tables.
     `define-abbrev-table' adds the new abbrev table name to this list.

 -- Function: insert-abbrev-table-description name &optional human
     This function inserts before point a description of the abbrev
     table named NAME.  The argument NAME is a symbol whose value is an
     abbrev table.   If HUMAN is non-`nil', the description is
     human-oriented.  System abbrevs are listed and identified as such.
     Otherwise the description is a Lisp expression--a call to
     `define-abbrev-table' that would define NAME as it is currently
     defined, but without the system abbrevs.  (The mode or package
     using NAME is supposed to add these to NAME separately.)


File: elisp,  Node: Defining Abbrevs,  Next: Abbrev Files,  Prev: Abbrev Tables,  Up: Abbrevs

36.2 Defining Abbrevs
=====================

`define-abbrev' is the low-level basic function for defining an abbrev
in an abbrev table.

   When a major mode defines a system abbrev, it should call
`define-abbrev' and specify `t' for the `:system' property.  Be aware
that any saved non-"system" abbrevs are restored at startup, i.e.
before some major modes are loaded.  Therefore, major modes should not
assume that their abbrev tables are empty when they are first loaded.

 -- Function: define-abbrev abbrev-table name expansion &optional hook
          &rest props
     This function defines an abbrev named NAME, in ABBREV-TABLE, to
     expand to EXPANSION and call HOOK, with properties PROPS (*note
     Abbrev Properties::).  The return value is NAME.  The `:system'
     property in PROPS is treated specially here: if it has the value
     `force', then it will overwrite an existing definition even for a
     non-"system" abbrev of the same name.

     NAME should be a string.  The argument EXPANSION is normally the
     desired expansion (a string), or `nil' to undefine the abbrev.  If
     it is anything but a string or `nil', then the abbreviation
     "expands" solely by running HOOK.

     The argument HOOK is a function or `nil'.  If HOOK is non-`nil',
     then it is called with no arguments after the abbrev is replaced
     with EXPANSION; point is located at the end of EXPANSION when HOOK
     is called.

     If HOOK is a non-`nil' symbol whose `no-self-insert' property is
     non-`nil', HOOK can explicitly control whether to insert the
     self-inserting input character that triggered the expansion.  If
     HOOK returns non-`nil' in this case, that inhibits insertion of
     the character.  By contrast, if HOOK returns `nil',
     `expand-abbrev' (or `abbrev-insert') also returns `nil', as if
     expansion had not really occurred.

     Normally, `define-abbrev' sets the variable `abbrevs-changed' to
     `t', if it actually changes the abbrev.  This is so that some
     commands will offer to save the abbrevs.  It does not do this for
     a system abbrev, since those aren't saved anyway.

 -- User Option: only-global-abbrevs
     If this variable is non-`nil', it means that the user plans to use
     global abbrevs only.  This tells the commands that define
     mode-specific abbrevs to define global ones instead.  This
     variable does not alter the behavior of the functions in this
     section; it is examined by their callers.


File: elisp,  Node: Abbrev Files,  Next: Abbrev Expansion,  Prev: Defining Abbrevs,  Up: Abbrevs

36.3 Saving Abbrevs in Files
============================

A file of saved abbrev definitions is actually a file of Lisp code.
The abbrevs are saved in the form of a Lisp program to define the same
abbrev tables with the same contents.  Therefore, you can load the file
with `load' (*note How Programs Do Loading::).  However, the function
`quietly-read-abbrev-file' is provided as a more convenient interface.
Emacs automatically calls this function at startup.

   User-level facilities such as `save-some-buffers' can save abbrevs
in a file automatically, under the control of variables described here.

 -- User Option: abbrev-file-name
     This is the default file name for reading and saving abbrevs.

 -- Function: quietly-read-abbrev-file &optional filename
     This function reads abbrev definitions from a file named FILENAME,
     previously written with `write-abbrev-file'.  If FILENAME is
     omitted or `nil', the file specified in `abbrev-file-name' is used.

     As the name implies, this function does not display any messages.

 -- User Option: save-abbrevs
     A non-`nil' value for `save-abbrevs' means that Emacs should offer
     to save abbrevs (if any have changed) when files are saved.  If
     the value is `silently', Emacs saves the abbrevs without asking
     the user.  `abbrev-file-name' specifies the file to save the
     abbrevs in.

 -- Variable: abbrevs-changed
     This variable is set non-`nil' by defining or altering any abbrevs
     (except system abbrevs).  This serves as a flag for various Emacs
     commands to offer to save your abbrevs.

 -- Command: write-abbrev-file &optional filename
     Save all abbrev definitions (except system abbrevs), for all abbrev
     tables listed in `abbrev-table-name-list', in the file FILENAME,
     in the form of a Lisp program that when loaded will define the
     same abbrevs.  If FILENAME is `nil' or omitted, `abbrev-file-name'
     is used.  This function returns `nil'.


File: elisp,  Node: Abbrev Expansion,  Next: Standard Abbrev Tables,  Prev: Abbrev Files,  Up: Abbrevs

36.4 Looking Up and Expanding Abbreviations
===========================================

Abbrevs are usually expanded by certain interactive commands, including
`self-insert-command'.  This section describes the subroutines used in
writing such commands, as well as the variables they use for
communication.

 -- Function: abbrev-symbol abbrev &optional table
     This function returns the symbol representing the abbrev named
     ABBREV.  It returns `nil' if that abbrev is not defined.  The
     optional second argument TABLE is the abbrev table in which to
     look it up.  If TABLE is `nil', this function tries first the
     current buffer's local abbrev table, and second the global abbrev
     table.

 -- Function: abbrev-expansion abbrev &optional table
     This function returns the string that ABBREV would expand into (as
     defined by the abbrev tables used for the current buffer).  It
     returns `nil' if ABBREV is not a valid abbrev.  The optional
     argument TABLE specifies the abbrev table to use, as in
     `abbrev-symbol'.

 -- Command: expand-abbrev
     This command expands the abbrev before point, if any.  If point
     does not follow an abbrev, this command does nothing.  The command
     returns the abbrev symbol if it did expansion, `nil' otherwise.

     If the abbrev symbol has a hook function that is a symbol whose
     `no-self-insert' property is non-`nil', and if the hook function
     returns `nil' as its value, then `expand-abbrev' returns `nil'
     even though expansion did occur.

 -- Function: abbrev-insert abbrev &optional name start end
     This function inserts the abbrev expansion of `abbrev', replacing
     the text between `start' and `end'.  If `start' is omitted, it
     defaults to point.  `name', if non-`nil', should be the name by
     which this abbrev was found (a string); it is used to figure out
     whether to adjust the capitalization of the expansion.  The
     function returns `abbrev' if the abbrev was successfully inserted.

 -- Command: abbrev-prefix-mark &optional arg
     This command marks the current location of point as the beginning
     of an abbrev.  The next call to `expand-abbrev' will use the text
     from here to point (where it is then) as the abbrev to expand,
     rather than using the previous word as usual.

     First, this command expands any abbrev before point, unless ARG is
     non-`nil'.  (Interactively, ARG is the prefix argument.)  Then it
     inserts a hyphen before point, to indicate the start of the next
     abbrev to be expanded.  The actual expansion removes the hyphen.

 -- User Option: abbrev-all-caps
     When this is set non-`nil', an abbrev entered entirely in upper
     case is expanded using all upper case.  Otherwise, an abbrev
     entered entirely in upper case is expanded by capitalizing each
     word of the expansion.

 -- Variable: abbrev-start-location
     The value of this variable is a buffer position (an integer or a
     marker) for `expand-abbrev' to use as the start of the next abbrev
     to be expanded.  The value can also be `nil', which means to use
     the word before point instead.  `abbrev-start-location' is set to
     `nil' each time `expand-abbrev' is called.  This variable is also
     set by `abbrev-prefix-mark'.

 -- Variable: abbrev-start-location-buffer
     The value of this variable is the buffer for which
     `abbrev-start-location' has been set.  Trying to expand an abbrev
     in any other buffer clears `abbrev-start-location'.  This variable
     is set by `abbrev-prefix-mark'.

 -- Variable: last-abbrev
     This is the `abbrev-symbol' of the most recent abbrev expanded.
     This information is left by `expand-abbrev' for the sake of the
     `unexpand-abbrev' command (*note Expanding Abbrevs:
     (emacs)Expanding Abbrevs.).

 -- Variable: last-abbrev-location
     This is the location of the most recent abbrev expanded.  This
     contains information left by `expand-abbrev' for the sake of the
     `unexpand-abbrev' command.

 -- Variable: last-abbrev-text
     This is the exact expansion text of the most recent abbrev
     expanded, after case conversion (if any).  Its value is `nil' if
     the abbrev has already been unexpanded.  This contains information
     left by `expand-abbrev' for the sake of the `unexpand-abbrev'
     command.

 -- Variable: abbrev-expand-functions
     This is a wrapper hook (*note Running Hooks::) run around the
     `expand-abbrev' function.  Each function on this hook is called
     with a single argument: a function that performs the normal abbrev
     expansion.  The hook function can hence do anything it wants before
     and after performing the expansion.  It can also choose not to call
     its argument, thus overriding the default behavior; or it may even
     call it several times.  The function should return the abbrev
     symbol if expansion took place.

   The following sample code shows a simple use of
`abbrev-expand-functions'.  It assumes that `foo-mode' is a mode for
editing certain files in which lines that start with `#' are comments.
You want to use Text mode abbrevs for those lines.  The regular local
abbrev table, `foo-mode-abbrev-table' is appropriate for all other
lines.  *Note Standard Abbrev Tables::, for the definitions of
`local-abbrev-table' and `text-mode-abbrev-table'.

     (defun foo-mode-abbrev-expand-function (expand)
       (if (not (save-excursion (forward-line 0) (eq (char-after) ?#)))
           ;; Performs normal expansion.
           (funcall expand)
         ;; We're inside a comment: use the text-mode abbrevs.
         (let ((local-abbrev-table text-mode-abbrev-table))
           (funcall expand))))

     (add-hook 'foo-mode-hook
               #'(lambda ()
                   (add-hook 'abbrev-expand-functions
                             'foo-mode-abbrev-expand-function
                             nil t)))


File: elisp,  Node: Standard Abbrev Tables,  Next: Abbrev Properties,  Prev: Abbrev Expansion,  Up: Abbrevs

36.5 Standard Abbrev Tables
===========================

Here we list the variables that hold the abbrev tables for the
preloaded major modes of Emacs.

 -- Variable: global-abbrev-table
     This is the abbrev table for mode-independent abbrevs.  The abbrevs
     defined in it apply to all buffers.  Each buffer may also have a
     local abbrev table, whose abbrev definitions take precedence over
     those in the global table.

 -- Variable: local-abbrev-table
     The value of this buffer-local variable is the (mode-specific)
     abbreviation table of the current buffer.  It can also be a list of
     such tables.

 -- Variable: abbrev-minor-mode-table-alist
     The value of this variable is a list of elements of the form
     `(MODE . ABBREV-TABLE)' where MODE is the name of a variable: if
     the variable is bound to a non-`nil' value, then the ABBREV-TABLE
     is active, otherwise it is ignored.  ABBREV-TABLE can also be a
     list of abbrev tables.

 -- Variable: fundamental-mode-abbrev-table
     This is the local abbrev table used in Fundamental mode; in other
     words, it is the local abbrev table in all buffers in Fundamental
     mode.

 -- Variable: text-mode-abbrev-table
     This is the local abbrev table used in Text mode.

 -- Variable: lisp-mode-abbrev-table
     This is the local abbrev table used in Lisp mode.  It is the parent
     of the local abbrev table used in Emacs Lisp mode.  *Note Abbrev
     Table Properties::.


File: elisp,  Node: Abbrev Properties,  Next: Abbrev Table Properties,  Prev: Standard Abbrev Tables,  Up: Abbrevs

36.6 Abbrev Properties
======================

Abbrevs have properties, some of which influence the way they work.
You can provide them as arguments to `define-abbrev', and manipulate
them with the following functions:

 -- Function: abbrev-put abbrev prop val
     Set the property PROP of ABBREV to value VAL.

 -- Function: abbrev-get abbrev prop
     Return the property PROP of ABBREV, or `nil' if the abbrev has no
     such property.

   The following properties have special meanings:

`:count'
     This property counts the number of times the abbrev has been
     expanded.  If not explicitly set, it is initialized to 0 by
     `define-abbrev'.

`:system'
     If non-`nil', this property marks the abbrev as a system abbrev.
     Such abbrevs are not saved (*note Abbrev Files::).

`:enable-function'
     If non-`nil', this property should be a function of no arguments
     which returns `nil' if the abbrev should not be used and `t'
     otherwise.

`:case-fixed'
     If non-`nil', this property indicates that the case of the
     abbrev's name is significant and should only match a text with the
     same pattern of capitalization.  It also disables the code that
     modifies the capitalization of the expansion.


File: elisp,  Node: Abbrev Table Properties,  Prev: Abbrev Properties,  Up: Abbrevs

36.7 Abbrev Table Properties
============================

Like abbrevs, abbrev tables have properties, some of which influence
the way they work.  You can provide them as arguments to
`define-abbrev-table', and manipulate them with the functions:

 -- Function: abbrev-table-put table prop val
     Set the property PROP of abbrev table TABLE to value VAL.

 -- Function: abbrev-table-get table prop
     Return the property PROP of abbrev table TABLE, or `nil' if the
     abbrev has no such property.

   The following properties have special meaning:

`:enable-function'
     This is like the `:enable-function' abbrev property except that it
     applies to all abbrevs in the table.  It is used before even trying
     to find the abbrev before point, so it can dynamically modify the
     abbrev table.

`:case-fixed'
     This is like the `:case-fixed' abbrev property except that it
     applies to all abbrevs in the table.

`:regexp'
     If non-`nil', this property is a regular expression that indicates
     how to extract the name of the abbrev before point, before looking
     it up in the table.  When the regular expression matches before
     point, the abbrev name is expected to be in submatch 1.  If this
     property is `nil', the default is to use `backward-word' and
     `forward-word' to find the name.  This property allows the use of
     abbrevs whose name contains characters of non-word syntax.

`:parents'
     This property holds a list of tables from which to inherit other
     abbrevs.

`:abbrev-table-modiff'
     This property holds a counter incremented each time a new abbrev is
     added to the table.



File: elisp,  Node: Processes,  Next: Display,  Prev: Abbrevs,  Up: Top

37 Processes
************

In the terminology of operating systems, a "process" is a space in
which a program can execute.  Emacs runs in a process.  Emacs Lisp
programs can invoke other programs in processes of their own.  These are
called "subprocesses" or "child processes" of the Emacs process, which
is their "parent process".

   A subprocess of Emacs may be "synchronous" or "asynchronous",
depending on how it is created.  When you create a synchronous
subprocess, the Lisp program waits for the subprocess to terminate
before continuing execution.  When you create an asynchronous
subprocess, it can run in parallel with the Lisp program.  This kind of
subprocess is represented within Emacs by a Lisp object which is also
called a "process".  Lisp programs can use this object to communicate
with the subprocess or to control it.  For example, you can send
signals, obtain status information, receive output from the process, or
send input to it.

 -- Function: processp object
     This function returns `t' if OBJECT represents an Emacs
     subprocess, `nil' otherwise.

   In addition to subprocesses of the current Emacs session, you can
also access other processes running on your machine.  *Note System
Processes::.

* Menu:

* Subprocess Creation::      Functions that start subprocesses.
* Shell Arguments::          Quoting an argument to pass it to a shell.
* Synchronous Processes::    Details of using synchronous subprocesses.
* Asynchronous Processes::   Starting up an asynchronous subprocess.
* Deleting Processes::       Eliminating an asynchronous subprocess.
* Process Information::      Accessing run-status and other attributes.
* Input to Processes::       Sending input to an asynchronous subprocess.
* Signals to Processes::     Stopping, continuing or interrupting
                               an asynchronous subprocess.
* Output from Processes::    Collecting output from an asynchronous subprocess.
* Sentinels::                Sentinels run when process run-status changes.
* Query Before Exit::        Whether to query if exiting will kill a process.
* System Processes::         Accessing other processes running on your system.
* Transaction Queues::       Transaction-based communication with subprocesses.
* Network::                  Opening network connections.
* Network Servers::          Network servers let Emacs accept net connections.
* Datagrams::                UDP network connections.
* Low-Level Network::        Lower-level but more general function
                               to create connections and servers.
* Misc Network::             Additional relevant functions for net connections.
* Serial Ports::             Communicating with serial ports.
* Byte Packing::             Using bindat to pack and unpack binary data.


File: elisp,  Node: Subprocess Creation,  Next: Shell Arguments,  Up: Processes

37.1 Functions that Create Subprocesses
=======================================

There are three primitives that create a new subprocess in which to run
a program.  One of them, `start-process', creates an asynchronous
process and returns a process object (*note Asynchronous Processes::).
The other two, `call-process' and `call-process-region', create a
synchronous process and do not return a process object (*note
Synchronous Processes::).  There are various higher-level functions
that make use of these primitives to run particular types of process.

   Synchronous and asynchronous processes are explained in the following
sections.  Since the three functions are all called in a similar
fashion, their common arguments are described here.

   In all cases, the function's PROGRAM argument specifies the program
to be run.  An error is signaled if the file is not found or cannot be
executed.  If the file name is relative, the variable `exec-path'
contains a list of directories to search.  Emacs initializes
`exec-path' when it starts up, based on the value of the environment
variable `PATH'.  The standard file name constructs, `~', `.', and
`..', are interpreted as usual in `exec-path', but environment variable
substitutions (`$HOME', etc.) are not recognized; use
`substitute-in-file-name' to perform them (*note File Name
Expansion::).  `nil' in this list refers to `default-directory'.

   Executing a program can also try adding suffixes to the specified
name:

 -- Variable: exec-suffixes
     This variable is a list of suffixes (strings) to try adding to the
     specified program file name.  The list should include `""' if you
     want the name to be tried exactly as specified.  The default value
     is system-dependent.

   *Please note:* The argument PROGRAM contains only the name of the
program; it may not contain any command-line arguments.  You must use a
separate argument, ARGS, to provide those, as described below.

   Each of the subprocess-creating functions has a BUFFER-OR-NAME
argument that specifies where the standard output from the program will
go.  It should be a buffer or a buffer name; if it is a buffer name,
that will create the buffer if it does not already exist.  It can also
be `nil', which says to discard the output unless a filter function
handles it.  (*Note Filter Functions::, and *note Read and Print::.)
Normally, you should avoid having multiple processes send output to the
same buffer because their output would be intermixed randomly.  For
synchronous processes, you can send the output to a file instead of a
buffer.

   All three of the subprocess-creating functions have a `&rest'
argument, ARGS.  The ARGS must all be strings, and they are supplied to
PROGRAM as separate command line arguments.  Wildcard characters and
other shell constructs have no special meanings in these strings, since
the strings are passed directly to the specified program.

   The subprocess inherits its environment from Emacs, but you can
specify overrides for it with `process-environment'.  *Note System
Environment::.  The subprocess gets its current directory from the
value of `default-directory'.

 -- Variable: exec-directory
     The value of this variable is a string, the name of a directory
     that contains programs that come with GNU Emacs and are intended
     for Emacs to invoke.  The program `movemail' is an example of such
     a program; Rmail uses it to fetch new mail from an inbox.

 -- User Option: exec-path
     The value of this variable is a list of directories to search for
     programs to run in subprocesses.  Each element is either the name
     of a directory (i.e., a string), or `nil', which stands for the
     default directory (which is the value of `default-directory').  

     The value of `exec-path' is used by `call-process' and
     `start-process' when the PROGRAM argument is not an absolute file
     name.

     Generally, you should not modify `exec-path' directly.  Instead,
     ensure that your `PATH' environment variable is set appropriately
     before starting Emacs.  Trying to modify `exec-path' independently
     of `PATH' can lead to confusing results.


File: elisp,  Node: Shell Arguments,  Next: Synchronous Processes,  Prev: Subprocess Creation,  Up: Processes

37.2 Shell Arguments
====================

Lisp programs sometimes need to run a shell and give it a command that
contains file names that were specified by the user.  These programs
ought to be able to support any valid file name.  But the shell gives
special treatment to certain characters, and if these characters occur
in the file name, they will confuse the shell.  To handle these
characters, use the function `shell-quote-argument':

 -- Function: shell-quote-argument argument
     This function returns a string that represents, in shell syntax,
     an argument whose actual contents are ARGUMENT.  It should work
     reliably to concatenate the return value into a shell command and
     then pass it to a shell for execution.

     Precisely what this function does depends on your operating
     system.  The function is designed to work with the syntax of your
     system's standard shell; if you use an unusual shell, you will
     need to redefine this function.

          ;; This example shows the behavior on GNU and Unix systems.
          (shell-quote-argument "foo > bar")
               => "foo\\ \\>\\ bar"

          ;; This example shows the behavior on MS-DOS and MS-Windows.
          (shell-quote-argument "foo > bar")
               => "\"foo > bar\""

     Here's an example of using `shell-quote-argument' to construct a
     shell command:

          (concat "diff -c "
                  (shell-quote-argument oldfile)
                  " "
                  (shell-quote-argument newfile))

   The following two functions are useful for combining a list of
individual command-line argument strings into a single string, and
taking a string apart into a list of individual command-line arguments.
These functions are mainly intended for converting user input in the
minibuffer, a Lisp string, into a list of string arguments to be passed
to `call-process' or `start-process', or for converting such lists of
arguments into a single Lisp string to be presented in the minibuffer
or echo area.

 -- Function: split-string-and-unquote string &optional separators
     This function splits STRING into substrings at matches for the
     regular expression SEPARATORS, like `split-string' does (*note
     Creating Strings::); in addition, it removes quoting from the
     substrings.  It then makes a list of the substrings and returns it.

     If SEPARATORS is omitted or `nil', it defaults to `"\\s-+"', which
     is a regular expression that matches one or more characters with
     whitespace syntax (*note Syntax Class Table::).

     This function supports two types of quoting: enclosing a whole
     string in double quotes `"..."', and quoting individual characters
     with a backslash escape `\'.  The latter is also used in Lisp
     strings, so this function can handle those as well.

 -- Function: combine-and-quote-strings list-of-strings &optional
          separator
     This function concatenates LIST-OF-STRINGS into a single string,
     quoting each string as necessary.  It also sticks the SEPARATOR
     string between each pair of strings; if SEPARATOR is omitted or
     `nil', it defaults to `" "'.  The return value is the resulting
     string.

     The strings in LIST-OF-STRINGS that need quoting are those that
     include SEPARATOR as their substring.  Quoting a string encloses
     it in double quotes `"..."'.  In the simplest case, if you are
     consing a command from the individual command-line arguments,
     every argument that includes embedded blanks will be quoted.


File: elisp,  Node: Synchronous Processes,  Next: Asynchronous Processes,  Prev: Shell Arguments,  Up: Processes

37.3 Creating a Synchronous Process
===================================

After a "synchronous process" is created, Emacs waits for the process
to terminate before continuing.  Starting Dired on GNU or Unix(1) is an
example of this: it runs `ls' in a synchronous process, then modifies
the output slightly.  Because the process is synchronous, the entire
directory listing arrives in the buffer before Emacs tries to do
anything with it.

   While Emacs waits for the synchronous subprocess to terminate, the
user can quit by typing `C-g'.  The first `C-g' tries to kill the
subprocess with a `SIGINT' signal; but it waits until the subprocess
actually terminates before quitting.  If during that time the user
types another `C-g', that kills the subprocess instantly with `SIGKILL'
and quits immediately (except on MS-DOS, where killing other processes
doesn't work).  *Note Quitting::.

   The synchronous subprocess functions return an indication of how the
process terminated.

   The output from a synchronous subprocess is generally decoded using a
coding system, much like text read from a file.  The input sent to a
subprocess by `call-process-region' is encoded using a coding system,
much like text written into a file.  *Note Coding Systems::.

 -- Function: call-process program &optional infile destination display
          &rest args
     This function calls PROGRAM and waits for it to finish.

     The standard input for the new process comes from file INFILE if
     INFILE is not `nil', and from the null device otherwise.  The
     argument DESTINATION says where to put the process output.  Here
     are the possibilities:

    a buffer
          Insert the output in that buffer, before point.  This
          includes both the standard output stream and the standard
          error stream of the process.

    a string
          Insert the output in a buffer with that name, before point.

    `t'
          Insert the output in the current buffer, before point.

    `nil'
          Discard the output.

    0
          Discard the output, and return `nil' immediately without
          waiting for the subprocess to finish.

          In this case, the process is not truly synchronous, since it
          can run in parallel with Emacs; but you can think of it as
          synchronous in that Emacs is essentially finished with the
          subprocess as soon as this function returns.

          MS-DOS doesn't support asynchronous subprocesses, so this
          option doesn't work there.

    `(:file FILE-NAME)'
          Send the output to the file name specified, overwriting it if
          it already exists.

    `(REAL-DESTINATION ERROR-DESTINATION)'
          Keep the standard output stream separate from the standard
          error stream; deal with the ordinary output as specified by
          REAL-DESTINATION, and dispose of the error output according
          to ERROR-DESTINATION.  If ERROR-DESTINATION is `nil', that
          means to discard the error output, `t' means mix it with the
          ordinary output, and a string specifies a file name to
          redirect error output into.

          You can't directly specify a buffer to put the error output
          in; that is too difficult to implement.  But you can achieve
          this result by sending the error output to a temporary file
          and then inserting the file into a buffer.

     If DISPLAY is non-`nil', then `call-process' redisplays the buffer
     as output is inserted.  (However, if the coding system chosen for
     decoding output is `undecided', meaning deduce the encoding from
     the actual data, then redisplay sometimes cannot continue once
     non-ASCII characters are encountered.  There are fundamental
     reasons why it is hard to fix this; see *note Output from
     Processes::.)

     Otherwise the function `call-process' does no redisplay, and the
     results become visible on the screen only when Emacs redisplays
     that buffer in the normal course of events.

     The remaining arguments, ARGS, are strings that specify command
     line arguments for the program.

     The value returned by `call-process' (unless you told it not to
     wait) indicates the reason for process termination.  A number
     gives the exit status of the subprocess; 0 means success, and any
     other value means failure.  If the process terminated with a
     signal, `call-process' returns a string describing the signal.

     In the examples below, the buffer `foo' is current.

          (call-process "pwd" nil t)
               => 0

          ---------- Buffer: foo ----------
          /home/lewis/manual
          ---------- Buffer: foo ----------

          (call-process "grep" nil "bar" nil "lewis" "/etc/passwd")
               => 0

          ---------- Buffer: bar ----------
          lewis:x:1001:1001:Bil Lewis,,,,:/home/lewis:/bin/bash

          ---------- Buffer: bar ----------

     Here is an example of the use of `call-process', as used to be
     found in the definition of the `insert-directory' function:

          (call-process insert-directory-program nil t nil switches
                        (if full-directory-p
                            (concat (file-name-as-directory file) ".")
                          file))

 -- Function: process-file program &optional infile buffer display
          &rest args
     This function processes files synchronously in a separate process.
     It is similar to `call-process', but may invoke a file handler
     based on the value of the variable `default-directory', which
     specifies the current working directory of the subprocess.

     The arguments are handled in almost the same way as for
     `call-process', with the following differences:

     Some file handlers may not support all combinations and forms of
     the arguments INFILE, BUFFER, and DISPLAY.  For example, some file
     handlers might behave as if DISPLAY were `nil', regardless of the
     value actually passed.  As another example, some file handlers
     might not support separating standard output and error output by
     way of the BUFFER argument.

     If a file handler is invoked, it determines the program to run
     based on the first argument PROGRAM.  For instance, suppose that a
     handler for remote files is invoked.  Then the path that is used
     for searching for the program might be different from `exec-path'.

     The second argument INFILE may invoke a file handler.  The file
     handler could be different from the handler chosen for the
     `process-file' function itself.  (For example, `default-directory'
     could be on one remote host, and INFILE on a different remote
     host.  Or `default-directory' could be non-special, whereas INFILE
     is on a remote host.)

     If BUFFER is a list of the form `(REAL-DESTINATION
     ERROR-DESTINATION)', and ERROR-DESTINATION names a file, then the
     same remarks as for INFILE apply.

     The remaining arguments (ARGS) will be passed to the process
     verbatim.  Emacs is not involved in processing file names that are
     present in ARGS.  To avoid confusion, it may be best to avoid
     absolute file names in ARGS, but rather to specify all file names
     as relative to `default-directory'.  The function
     `file-relative-name' is useful for constructing such relative file
     names.

 -- Variable: process-file-side-effects
     This variable indicates whether a call of `process-file' changes
     remote files.

     By default, this variable is always set to `t', meaning that a
     call of `process-file' could potentially change any file on a
     remote host.  When set to `nil', a file handler could optimize its
     behavior with respect to remote file attribute caching.

     You should only ever change this variable with a let-binding; never
     with `setq'.

 -- Function: call-process-region start end program &optional delete
          destination display &rest args
     This function sends the text from START to END as standard input
     to a process running PROGRAM.  It deletes the text sent if DELETE
     is non-`nil'; this is useful when DESTINATION is `t', to insert
     the output in the current buffer in place of the input.

     The arguments DESTINATION and DISPLAY control what to do with the
     output from the subprocess, and whether to update the display as
     it comes in.  For details, see the description of `call-process',
     above.  If DESTINATION is the integer 0, `call-process-region'
     discards the output and returns `nil' immediately, without waiting
     for the subprocess to finish (this only works if asynchronous
     subprocesses are supported; i.e. not on MS-DOS).

     The remaining arguments, ARGS, are strings that specify command
     line arguments for the program.

     The return value of `call-process-region' is just like that of
     `call-process': `nil' if you told it to return without waiting;
     otherwise, a number or string which indicates how the subprocess
     terminated.

     In the following example, we use `call-process-region' to run the
     `cat' utility, with standard input being the first five characters
     in buffer `foo' (the word `input').  `cat' copies its standard
     input into its standard output.  Since the argument DESTINATION is
     `t', this output is inserted in the current buffer.

          ---------- Buffer: foo ----------
          input-!-
          ---------- Buffer: foo ----------

          (call-process-region 1 6 "cat" nil t)
               => 0

          ---------- Buffer: foo ----------
          inputinput-!-
          ---------- Buffer: foo ----------

     For example, the `shell-command-on-region' command uses
     `call-process-region' in a manner similar to this:

          (call-process-region
           start end
           shell-file-name      ; name of program
           nil                  ; do not delete region
           buffer               ; send output to `buffer'
           nil                  ; no redisplay during output
           "-c" command)        ; arguments for the shell

 -- Function: call-process-shell-command command &optional infile
          destination display &rest args
     This function executes the shell command COMMAND synchronously.
     The final arguments ARGS are additional arguments to add at the
     end of COMMAND.  The other arguments are handled as in
     `call-process'.

 -- Function: process-file-shell-command command &optional infile
          destination display &rest args
     This function is like `call-process-shell-command', but uses
     `process-file' internally.  Depending on `default-directory',
     COMMAND can be executed also on remote hosts.

 -- Function: shell-command-to-string command
     This function executes COMMAND (a string) as a shell command, then
     returns the command's output as a string.

 -- Function: process-lines program &rest args
     This function runs PROGRAM, waits for it to finish, and returns
     its output as a list of strings.  Each string in the list holds a
     single line of text output by the program; the end-of-line
     characters are stripped from each line.  The arguments beyond
     PROGRAM, ARGS, are strings that specify command-line arguments
     with which to run the program.

     If PROGRAM exits with a non-zero exit status, this function
     signals an error.

     This function works by calling `call-process', so program output
     is decoded in the same way as for `call-process'.

   ---------- Footnotes ----------

   (1) On other systems, Emacs uses a Lisp emulation of `ls'; see *note
Contents of Directories::.


File: elisp,  Node: Asynchronous Processes,  Next: Deleting Processes,  Prev: Synchronous Processes,  Up: Processes

37.4 Creating an Asynchronous Process
=====================================

After an "asynchronous process" is created, Emacs and the subprocess
both continue running immediately.  The process thereafter runs in
parallel with Emacs, and the two can communicate with each other using
the functions described in the following sections.  However,
communication is only partially asynchronous: Emacs sends data to the
process only when certain functions are called, and Emacs accepts data
from the process only when Emacs is waiting for input or for a time
delay.

   Here we describe how to create an asynchronous process.

 -- Function: start-process name buffer-or-name program &rest args
     This function creates a new asynchronous subprocess and starts the
     program PROGRAM running in it.  It returns a process object that
     stands for the new subprocess in Lisp.  The argument NAME
     specifies the name for the process object; if a process with this
     name already exists, then NAME is modified (by appending `<1>',
     etc.) to be unique.  The buffer BUFFER-OR-NAME is the buffer to
     associate with the process.

     If PROGRAM is `nil', Emacs opens a new pseudoterminal (pty) and
     associates its input and output with BUFFER-OR-NAME, without
     creating a subprocess.  In that case, the remaining arguments ARGS
     are ignored.

     The remaining arguments, ARGS, are strings that specify command
     line arguments for the subprocess.

     In the example below, the first process is started and runs
     (rather, sleeps) for 100 seconds (the output buffer `foo' is
     created immediately).  Meanwhile, the second process is started,
     and given the name `my-process<1>' for the sake of uniqueness.  It
     inserts the directory listing at the end of the buffer `foo',
     before the first process finishes.  Then it finishes, and a
     message to that effect is inserted in the buffer.  Much later, the
     first process finishes, and another message is inserted in the
     buffer for it.

          (start-process "my-process" "foo" "sleep" "100")
               => #<process my-process>

          (start-process "my-process" "foo" "ls" "-l" "/bin")
               => #<process my-process<1>>

          ---------- Buffer: foo ----------
          total 8336
          -rwxr-xr-x 1 root root 971384 Mar 30 10:14 bash
          -rwxr-xr-x 1 root root 146920 Jul  5  2011 bsd-csh
          ...
          -rwxr-xr-x 1 root root 696880 Feb 28 15:55 zsh4

          Process my-process<1> finished

          Process my-process finished
          ---------- Buffer: foo ----------

 -- Function: start-file-process name buffer-or-name program &rest args
     Like `start-process', this function starts a new asynchronous
     subprocess running PROGRAM in it, and returns its process object.

     The difference from `start-process' is that this function may
     invoked a file handler based on the value of `default-directory'.
     This handler ought to run PROGRAM, perhaps on the local host,
     perhaps on a remote host that corresponds to `default-directory'.
     In the latter case, the local part of `default-directory' becomes
     the working directory of the process.

     This function does not try to invoke file name handlers for
     PROGRAM or for the PROGRAM-ARGS.

     Depending on the implementation of the file handler, it might not
     be possible to apply `process-filter' or `process-sentinel' to the
     resulting process object.  *Note Filter Functions::, and *note
     Sentinels::.

     Some file handlers may not support `start-file-process' (for
     example the function `ange-ftp-hook-function').  In such cases,
     this function does nothing and returns `nil'.

 -- Function: start-process-shell-command name buffer-or-name command
     This function is like `start-process', except that it uses a shell
     to execute the specified command.  The argument COMMAND is a shell
     command name.  The variable `shell-file-name' specifies which
     shell to use.

     The point of running a program through the shell, rather than
     directly with `start-process', is so that you can employ shell
     features such as wildcards in the arguments.  It follows that if
     you include any arbitrary user-specified arguments in the command,
     you should quote them with `shell-quote-argument' first, so that
     any special shell characters do _not_ have their special shell
     meanings.  *Note Shell Arguments::.  Of course, when executing
     commands based on user input you should also consider the security
     implications.

 -- Function: start-file-process-shell-command name buffer-or-name
          command
     This function is like `start-process-shell-command', but uses
     `start-file-process' internally.  Because of this, COMMAND can
     also be executed on remote hosts, depending on `default-directory'.

 -- Variable: process-connection-type
     This variable controls the type of device used to communicate with
     asynchronous subprocesses.  If it is non-`nil', then PTYs are
     used, when available.  Otherwise, pipes are used.

     PTYs are usually preferable for processes visible to the user, as
     in Shell mode, because they allow job control (`C-c', `C-z', etc.)
     to work between the process and its children, whereas pipes do
     not.  For subprocesses used for internal purposes by programs, it
     is often better to use a pipe, because they are more efficient.  In
     addition, the total number of PTYs is limited on many systems and
     it is good not to waste them.

     The value of `process-connection-type' takes effect when
     `start-process' is called.  So you can specify how to communicate
     with one subprocess by binding the variable around the call to
     `start-process'.

          (let ((process-connection-type nil))  ; use a pipe
            (start-process ...))

     To determine whether a given subprocess actually got a pipe or a
     PTY, use the function `process-tty-name' (*note Process
     Information::).


File: elisp,  Node: Deleting Processes,  Next: Process Information,  Prev: Asynchronous Processes,  Up: Processes

37.5 Deleting Processes
=======================

"Deleting a process" disconnects Emacs immediately from the subprocess.
Processes are deleted automatically after they terminate, but not
necessarily right away.  You can delete a process explicitly at any
time.  If you explicitly delete a terminated process before it is
deleted automatically, no harm results.  Deleting a running process
sends a signal to terminate it (and its child processes, if any), and
calls the process sentinel if it has one.  *Note Sentinels::.

   When a process is deleted, the process object itself continues to
exist as long as other Lisp objects point to it.  All the Lisp
primitives that work on process objects accept deleted processes, but
those that do I/O or send signals will report an error.  The process
mark continues to point to the same place as before, usually into a
buffer where output from the process was being inserted.

 -- User Option: delete-exited-processes
     This variable controls automatic deletion of processes that have
     terminated (due to calling `exit' or to a signal).  If it is
     `nil', then they continue to exist until the user runs
     `list-processes'.  Otherwise, they are deleted immediately after
     they exit.

 -- Function: delete-process process
     This function deletes a process, killing it with a `SIGKILL'
     signal.  The argument may be a process, the name of a process, a
     buffer, or the name of a buffer.  (A buffer or buffer-name stands
     for the process that `get-buffer-process' returns.)  Calling
     `delete-process' on a running process terminates it, updates the
     process status, and runs the sentinel (if any) immediately.  If the
     process has already terminated, calling `delete-process' has no
     effect on its status, or on the running of its sentinel (which will
     happen sooner or later).

          (delete-process "*shell*")
               => nil


File: elisp,  Node: Process Information,  Next: Input to Processes,  Prev: Deleting Processes,  Up: Processes

37.6 Process Information
========================

Several functions return information about processes.

 -- Command: list-processes &optional query-only buffer
     This command displays a listing of all living processes.  In
     addition, it finally deletes any process whose status was `Exited'
     or `Signaled'.  It returns `nil'.

     The processes are shown in a buffer named `*Process List*' (unless
     you specify otherwise using the optional argument BUFFER), whose
     major mode is Process Menu mode.

     If QUERY-ONLY is non-`nil', it only lists processes whose query
     flag is non-`nil'.  *Note Query Before Exit::.

 -- Function: process-list
     This function returns a list of all processes that have not been
     deleted.

          (process-list)
               => (#<process display-time> #<process shell>)

 -- Function: get-process name
     This function returns the process named NAME (a string), or `nil'
     if there is none.

          (get-process "shell")
               => #<process shell>

 -- Function: process-command process
     This function returns the command that was executed to start
     PROCESS.  This is a list of strings, the first string being the
     program executed and the rest of the strings being the arguments
     that were given to the program.

          (process-command (get-process "shell"))
               => ("bash" "-i")

 -- Function: process-contact process &optional key
     This function returns information about how a network or serial
     process was set up.  When KEY is `nil', it returns `(HOSTNAME
     SERVICE)' for a network process, and `(PORT SPEED)' for a serial
     process.  For an ordinary child process, this function always
     returns `t'.

     If KEY is `t', the value is the complete status information for
     the connection, server, or serial port; that is, the list of
     keywords and values specified in `make-network-process' or
     `make-serial-process', except that some of the values represent
     the current status instead of what you specified.

     For a network process, the values include (see
     `make-network-process' for a complete list):

    `:buffer'
          The associated value is the process buffer.

    `:filter'
          The associated value is the process filter function.

    `:sentinel'
          The associated value is the process sentinel function.

    `:remote'
          In a connection, the address in internal format of the remote
          peer.

    `:local'
          The local address, in internal format.

    `:service'
          In a server, if you specified `t' for SERVICE, this value is
          the actual port number.

     `:local' and `:remote' are included even if they were not
     specified explicitly in `make-network-process'.

     For a serial process, see `make-serial-process' and
     `serial-process-configure' for a list of keys.

     If KEY is a keyword, the function returns the value corresponding
     to that keyword.

 -- Function: process-id process
     This function returns the PID of PROCESS.  This is an integer that
     distinguishes the process PROCESS from all other processes running
     on the same computer at the current time.  The PID of a process is
     chosen by the operating system kernel when the process is started
     and remains constant as long as the process exists.

 -- Function: process-name process
     This function returns the name of PROCESS, as a string.

 -- Function: process-status process-name
     This function returns the status of PROCESS-NAME as a symbol.  The
     argument PROCESS-NAME must be a process, a buffer, or a process
     name (a string).

     The possible values for an actual subprocess are:

    `run'
          for a process that is running.

    `stop'
          for a process that is stopped but continuable.

    `exit'
          for a process that has exited.

    `signal'
          for a process that has received a fatal signal.

    `open'
          for a network connection that is open.

    `closed'
          for a network connection that is closed.  Once a connection
          is closed, you cannot reopen it, though you might be able to
          open a new connection to the same place.

    `connect'
          for a non-blocking connection that is waiting to complete.

    `failed'
          for a non-blocking connection that has failed to complete.

    `listen'
          for a network server that is listening.

    `nil'
          if PROCESS-NAME is not the name of an existing process.

          (process-status (get-buffer "*shell*"))
               => run

     For a network connection, `process-status' returns one of the
     symbols `open' or `closed'.  The latter means that the other side
     closed the connection, or Emacs did `delete-process'.

 -- Function: process-live-p process
     This function returns non-`nil' if PROCESS is alive.  A process is
     considered alive if its status is `run', `open', `listen',
     `connect' or `stop'.

 -- Function: process-type process
     This function returns the symbol `network' for a network
     connection or server, `serial' for a serial port connection, or
     `real' for a real subprocess.

 -- Function: process-exit-status process
     This function returns the exit status of PROCESS or the signal
     number that killed it.  (Use the result of `process-status' to
     determine which of those it is.)  If PROCESS has not yet
     terminated, the value is 0.

 -- Function: process-tty-name process
     This function returns the terminal name that PROCESS is using for
     its communication with Emacs--or `nil' if it is using pipes
     instead of a terminal (see `process-connection-type' in *note
     Asynchronous Processes::).  If PROCESS represents a program
     running on a remote host, the terminal name used by that program on
     the remote host is provided as process property `remote-tty'.

 -- Function: process-coding-system process
     This function returns a cons cell `(DECODE . ENCODE)', describing
     the coding systems in use for decoding output from, and encoding
     input to, PROCESS (*note Coding Systems::).

 -- Function: set-process-coding-system process &optional
          decoding-system encoding-system
     This function specifies the coding systems to use for subsequent
     output from and input to PROCESS.  It will use DECODING-SYSTEM to
     decode subprocess output, and ENCODING-SYSTEM to encode subprocess
     input.

   Every process also has a property list that you can use to store
miscellaneous values associated with the process.

 -- Function: process-get process propname
     This function returns the value of the PROPNAME property of
     PROCESS.

 -- Function: process-put process propname value
     This function sets the value of the PROPNAME property of PROCESS
     to VALUE.

 -- Function: process-plist process
     This function returns the process plist of PROCESS.

 -- Function: set-process-plist process plist
     This function sets the process plist of PROCESS to PLIST.


File: elisp,  Node: Input to Processes,  Next: Signals to Processes,  Prev: Process Information,  Up: Processes

37.7 Sending Input to Processes
===============================

Asynchronous subprocesses receive input when it is sent to them by
Emacs, which is done with the functions in this section.  You must
specify the process to send input to, and the input data to send.  The
data appears on the "standard input" of the subprocess.

   Some operating systems have limited space for buffered input in a
PTY.  On these systems, Emacs sends an EOF periodically amidst the
other characters, to force them through.  For most programs, these EOFs
do no harm.

   Subprocess input is normally encoded using a coding system before the
subprocess receives it, much like text written into a file.  You can use
`set-process-coding-system' to specify which coding system to use
(*note Process Information::).  Otherwise, the coding system comes from
`coding-system-for-write', if that is non-`nil'; or else from the
defaulting mechanism (*note Default Coding Systems::).

   Sometimes the system is unable to accept input for that process,
because the input buffer is full.  When this happens, the send functions
wait a short while, accepting output from subprocesses, and then try
again.  This gives the subprocess a chance to read more of its pending
input and make space in the buffer.  It also allows filters, sentinels
and timers to run--so take account of that in writing your code.

   In these functions, the PROCESS argument can be a process or the
name of a process, or a buffer or buffer name (which stands for a
process via `get-buffer-process').  `nil' means the current buffer's
process.

 -- Function: process-send-string process string
     This function sends PROCESS the contents of STRING as standard
     input.  It returns `nil'.  For example, to make a Shell buffer
     list files:

          (process-send-string "shell<1>" "ls\n")
               => nil

 -- Function: process-send-region process start end
     This function sends the text in the region defined by START and
     END as standard input to PROCESS.

     An error is signaled unless both START and END are integers or
     markers that indicate positions in the current buffer.  (It is
     unimportant which number is larger.)

 -- Function: process-send-eof &optional process
     This function makes PROCESS see an end-of-file in its input.  The
     EOF comes after any text already sent to it.  The function returns
     PROCESS.

          (process-send-eof "shell")
               => "shell"

 -- Function: process-running-child-p &optional process
     This function will tell you whether a PROCESS has given control of
     its terminal to its own child process.  The value is `t' if this is
     true, or if Emacs cannot tell; it is `nil' if Emacs can be certain
     that this is not so.


File: elisp,  Node: Signals to Processes,  Next: Output from Processes,  Prev: Input to Processes,  Up: Processes

37.8 Sending Signals to Processes
=================================

"Sending a signal" to a subprocess is a way of interrupting its
activities.  There are several different signals, each with its own
meaning.  The set of signals and their names is defined by the operating
system.  For example, the signal `SIGINT' means that the user has typed
`C-c', or that some analogous thing has happened.

   Each signal has a standard effect on the subprocess.  Most signals
kill the subprocess, but some stop (or resume) execution instead.  Most
signals can optionally be handled by programs; if the program handles
the signal, then we can say nothing in general about its effects.

   You can send signals explicitly by calling the functions in this
section.  Emacs also sends signals automatically at certain times:
killing a buffer sends a `SIGHUP' signal to all its associated
processes; killing Emacs sends a `SIGHUP' signal to all remaining
processes.  (`SIGHUP' is a signal that usually indicates that the user
"hung up the phone", i.e., disconnected.)

   Each of the signal-sending functions takes two optional arguments:
PROCESS and CURRENT-GROUP.

   The argument PROCESS must be either a process, a process name, a
buffer, a buffer name, or `nil'.  A buffer or buffer name stands for a
process through `get-buffer-process'.  `nil' stands for the process
associated with the current buffer.  An error is signaled if PROCESS
does not identify a process.

   The argument CURRENT-GROUP is a flag that makes a difference when
you are running a job-control shell as an Emacs subprocess.  If it is
non-`nil', then the signal is sent to the current process-group of the
terminal that Emacs uses to communicate with the subprocess.  If the
process is a job-control shell, this means the shell's current subjob.
If it is `nil', the signal is sent to the process group of the
immediate subprocess of Emacs.  If the subprocess is a job-control
shell, this is the shell itself.

   The flag CURRENT-GROUP has no effect when a pipe is used to
communicate with the subprocess, because the operating system does not
support the distinction in the case of pipes.  For the same reason,
job-control shells won't work when a pipe is used.  See
`process-connection-type' in *note Asynchronous Processes::.

 -- Function: interrupt-process &optional process current-group
     This function interrupts the process PROCESS by sending the signal
     `SIGINT'.  Outside of Emacs, typing the "interrupt character"
     (normally `C-c' on some systems, and <DEL> on others) sends this
     signal.  When the argument CURRENT-GROUP is non-`nil', you can
     think of this function as "typing `C-c'" on the terminal by which
     Emacs talks to the subprocess.

 -- Function: kill-process &optional process current-group
     This function kills the process PROCESS by sending the signal
     `SIGKILL'.  This signal kills the subprocess immediately, and
     cannot be handled by the subprocess.

 -- Function: quit-process &optional process current-group
     This function sends the signal `SIGQUIT' to the process PROCESS.
     This signal is the one sent by the "quit character" (usually `C-b'
     or `C-\') when you are not inside Emacs.

 -- Function: stop-process &optional process current-group
     This function stops the process PROCESS by sending the signal
     `SIGTSTP'.  Use `continue-process' to resume its execution.

     Outside of Emacs, on systems with job control, the "stop character"
     (usually `C-z') normally sends this signal.  When CURRENT-GROUP is
     non-`nil', you can think of this function as "typing `C-z'" on the
     terminal Emacs uses to communicate with the subprocess.

 -- Function: continue-process &optional process current-group
     This function resumes execution of the process PROCESS by sending
     it the signal `SIGCONT'.  This presumes that PROCESS was stopped
     previously.

 -- Command: signal-process process signal
     This function sends a signal to process PROCESS.  The argument
     SIGNAL specifies which signal to send; it should be an integer, or
     a symbol whose name is a signal.

     The PROCESS argument can be a system process ID (an integer); that
     allows you to send signals to processes that are not children of
     Emacs.  *Note System Processes::.


File: elisp,  Node: Output from Processes,  Next: Sentinels,  Prev: Signals to Processes,  Up: Processes

37.9 Receiving Output from Processes
====================================

There are two ways to receive the output that a subprocess writes to
its standard output stream.  The output can be inserted in a buffer,
which is called the associated buffer of the process (*note Process
Buffers::), or a function called the "filter function" can be called to
act on the output.  If the process has no buffer and no filter
function, its output is discarded.

   When a subprocess terminates, Emacs reads any pending output, then
stops reading output from that subprocess.  Therefore, if the
subprocess has children that are still live and still producing output,
Emacs won't receive that output.

   Output from a subprocess can arrive only while Emacs is waiting: when
reading terminal input (see the function `waiting-for-user-input-p'),
in `sit-for' and `sleep-for' (*note Waiting::), and in
`accept-process-output' (*note Accepting Output::).  This minimizes the
problem of timing errors that usually plague parallel programming.  For
example, you can safely create a process and only then specify its
buffer or filter function; no output can arrive before you finish, if
the code in between does not call any primitive that waits.

 -- Variable: process-adaptive-read-buffering
     On some systems, when Emacs reads the output from a subprocess, the
     output data is read in very small blocks, potentially resulting in
     very poor performance.  This behavior can be remedied to some
     extent by setting the variable `process-adaptive-read-buffering'
     to a non-`nil' value (the default), as it will automatically delay
     reading from such processes, thus allowing them to produce more
     output before Emacs tries to read it.

   It is impossible to separate the standard output and standard error
streams of the subprocess, because Emacs normally spawns the subprocess
inside a pseudo-TTY, and a pseudo-TTY has only one output channel.  If
you want to keep the output to those streams separate, you should
redirect one of them to a file--for example, by using an appropriate
shell command.

* Menu:

* Process Buffers::         If no filter, output is put in a buffer.
* Filter Functions::        Filter functions accept output from the process.
* Decoding Output::         Filters can get unibyte or multibyte strings.
* Accepting Output::        How to wait until process output arrives.


File: elisp,  Node: Process Buffers,  Next: Filter Functions,  Up: Output from Processes

37.9.1 Process Buffers
----------------------

A process can (and usually does) have an "associated buffer", which is
an ordinary Emacs buffer that is used for two purposes: storing the
output from the process, and deciding when to kill the process.  You
can also use the buffer to identify a process to operate on, since in
normal practice only one process is associated with any given buffer.
Many applications of processes also use the buffer for editing input to
be sent to the process, but this is not built into Emacs Lisp.

   Unless the process has a filter function (*note Filter Functions::),
its output is inserted in the associated buffer.  The position to insert
the output is determined by the `process-mark', which is then updated
to point to the end of the text just inserted.  Usually, but not
always, the `process-mark' is at the end of the buffer.

   Killing the associated buffer of a process also kills the process.
Emacs asks for confirmation first, if the process's
`process-query-on-exit-flag' is non-`nil' (*note Query Before Exit::).
This confirmation is done by the function
`process-kill-buffer-query-function', which is run from
`kill-buffer-query-functions' (*note Killing Buffers::).

 -- Function: process-buffer process
     This function returns the associated buffer of the process PROCESS.

          (process-buffer (get-process "shell"))
               => #<buffer *shell*>

 -- Function: process-mark process
     This function returns the process marker for PROCESS, which is the
     marker that says where to insert output from the process.

     If PROCESS does not have a buffer, `process-mark' returns a marker
     that points nowhere.

     Insertion of process output in a buffer uses this marker to decide
     where to insert, and updates it to point after the inserted text.
     That is why successive batches of output are inserted
     consecutively.

     Filter functions normally should use this marker in the same
     fashion as is done by direct insertion of output in the buffer.
     For an example of a filter function that uses `process-mark',
     *note Process Filter Example::.

     When the user is expected to enter input in the process buffer for
     transmission to the process, the process marker separates the new
     input from previous output.

 -- Function: set-process-buffer process buffer
     This function sets the buffer associated with PROCESS to BUFFER.
     If BUFFER is `nil', the process becomes associated with no buffer.

 -- Function: get-buffer-process buffer-or-name
     This function returns a nondeleted process associated with the
     buffer specified by BUFFER-OR-NAME.  If there are several processes
     associated with it, this function chooses one (currently, the one
     most recently created, but don't count on that).  Deletion of a
     process (see `delete-process') makes it ineligible for this
     function to return.

     It is usually a bad idea to have more than one process associated
     with the same buffer.

          (get-buffer-process "*shell*")
               => #<process shell>

     Killing the process's buffer deletes the process, which kills the
     subprocess with a `SIGHUP' signal (*note Signals to Processes::).


File: elisp,  Node: Filter Functions,  Next: Decoding Output,  Prev: Process Buffers,  Up: Output from Processes

37.9.2 Process Filter Functions
-------------------------------

A process "filter function" is a function that receives the standard
output from the associated process.  If a process has a filter, then
_all_ output from that process is passed to the filter.  The process
buffer is used directly for output from the process only when there is
no filter.

   The filter function can only be called when Emacs is waiting for
something, because process output arrives only at such times.  Emacs
waits when reading terminal input (see the function
`waiting-for-user-input-p'), in `sit-for' and `sleep-for' (*note
Waiting::), and in `accept-process-output' (*note Accepting Output::).

   A filter function must accept two arguments: the associated process
and a string, which is output just received from it.  The function is
then free to do whatever it chooses with the output.

   Quitting is normally inhibited within a filter function--otherwise,
the effect of typing `C-g' at command level or to quit a user command
would be unpredictable.  If you want to permit quitting inside a filter
function, bind `inhibit-quit' to `nil'.  In most cases, the right way
to do this is with the macro `with-local-quit'.  *Note Quitting::.

   If an error happens during execution of a filter function, it is
caught automatically, so that it doesn't stop the execution of whatever
program was running when the filter function was started.  However, if
`debug-on-error' is non-`nil', errors are not caught.  This makes it
possible to use the Lisp debugger to debug the filter function.  *Note
Debugger::.

   Many filter functions sometimes (or always) insert the output in the
process's buffer, mimicking the actions of Emacs when there is no
filter.  Such filter functions need to make sure that they save the
current buffer, select the correct buffer (if different) before
inserting output, and then restore the original buffer.  They should
also check whether the buffer is still alive, update the process
marker, and in some cases update the value of point.  Here is how to do
these things:

     (defun ordinary-insertion-filter (proc string)
       (when (buffer-live-p (process-buffer proc))
         (with-current-buffer (process-buffer proc)
           (let ((moving (= (point) (process-mark proc))))
             (save-excursion
               ;; Insert the text, advancing the process marker.
               (goto-char (process-mark proc))
               (insert string)
               (set-marker (process-mark proc) (point)))
             (if moving (goto-char (process-mark proc)))))))

   To make the filter force the process buffer to be visible whenever
new text arrives, you could insert a line like the following just
before the `with-current-buffer' construct:

     (display-buffer (process-buffer proc))

   To force point to the end of the new output, no matter where it was
previously, eliminate the variable `moving' and call `goto-char'
unconditionally.

   Note that Emacs automatically saves and restores the match data
while executing filter functions.  *Note Match Data::.

   The output to the filter may come in chunks of any size.  A program
that produces the same output twice in a row may send it as one batch of
200 characters one time, and five batches of 40 characters the next.  If
the filter looks for certain text strings in the subprocess output, make
sure to handle the case where one of these strings is split across two
or more batches of output; one way to do this is to insert the received
text into a temporary buffer, which can then be searched.

 -- Function: set-process-filter process filter
     This function gives PROCESS the filter function FILTER.  If FILTER
     is `nil', it gives the process no filter.

 -- Function: process-filter process
     This function returns the filter function of PROCESS, or `nil' if
     it has none.

   Here is an example of the use of a filter function:

     (defun keep-output (process output)
        (setq kept (cons output kept)))
          => keep-output
     (setq kept nil)
          => nil
     (set-process-filter (get-process "shell") 'keep-output)
          => keep-output
     (process-send-string "shell" "ls ~/other\n")
          => nil
     kept
          => ("lewis@slug:$ "
     "FINAL-W87-SHORT.MSS    backup.otl              kolstad.mss~
     address.txt             backup.psf              kolstad.psf
     backup.bib~             david.mss               resume-Dec-86.mss~
     backup.err              david.psf               resume-Dec.psf
     backup.mss              dland                   syllabus.mss
     "
     "#backups.mss#          backup.mss~             kolstad.mss
     ")


File: elisp,  Node: Decoding Output,  Next: Accepting Output,  Prev: Filter Functions,  Up: Output from Processes

37.9.3 Decoding Process Output
------------------------------

When Emacs writes process output directly into a multibyte buffer, it
decodes the output according to the process output coding system.  If
the coding system is `raw-text' or `no-conversion', Emacs converts the
unibyte output to multibyte using `string-to-multibyte', and inserts
the resulting multibyte text.

   You can use `set-process-coding-system' to specify which coding
system to use (*note Process Information::).  Otherwise, the coding
system comes from `coding-system-for-read', if that is non-`nil'; or
else from the defaulting mechanism (*note Default Coding Systems::).
If the text output by a process contains null bytes, Emacs by default
uses `no-conversion' for it; see *note inhibit-null-byte-detection:
Lisp and Coding Systems, for how to control this behavior.

   *Warning:* Coding systems such as `undecided', which determine the
coding system from the data, do not work entirely reliably with
asynchronous subprocess output.  This is because Emacs has to process
asynchronous subprocess output in batches, as it arrives.  Emacs must
try to detect the proper coding system from one batch at a time, and
this does not always work.  Therefore, if at all possible, specify a
coding system that determines both the character code conversion and
the end of line conversion--that is, one like `latin-1-unix', rather
than `undecided' or `latin-1'.

   When Emacs calls a process filter function, it provides the process
output as a multibyte string or as a unibyte string according to the
process's filter coding system.  Emacs decodes the output according to
the process output coding system, which usually produces a multibyte
string, except for coding systems such as `binary' and `raw-text'.


File: elisp,  Node: Accepting Output,  Prev: Decoding Output,  Up: Output from Processes

37.9.4 Accepting Output from Processes
--------------------------------------

Output from asynchronous subprocesses normally arrives only while Emacs
is waiting for some sort of external event, such as elapsed time or
terminal input.  Occasionally it is useful in a Lisp program to
explicitly permit output to arrive at a specific point, or even to wait
until output arrives from a process.

 -- Function: accept-process-output &optional process seconds millisec
          just-this-one
     This function allows Emacs to read pending output from processes.
     The output is inserted in the associated buffers or given to their
     filter functions.  If PROCESS is non-`nil' then this function does
     not return until some output has been received from PROCESS.

     The arguments SECONDS and MILLISEC let you specify timeout
     periods.  The former specifies a period measured in seconds and the
     latter specifies one measured in milliseconds.  The two time
     periods thus specified are added together, and
     `accept-process-output' returns after that much time, whether or
     not there has been any subprocess output.

     The argument MILLISEC is obsolete (and should not be used),
     because SECONDS can be a floating point number to specify waiting
     a fractional number of seconds.  If SECONDS is 0, the function
     accepts whatever output is pending but does not wait.

     If PROCESS is a process, and the argument JUST-THIS-ONE is
     non-`nil', only output from that process is handled, suspending
     output from other processes until some output has been received
     from that process or the timeout expires.  If JUST-THIS-ONE is an
     integer, also inhibit running timers.  This feature is generally
     not recommended, but may be necessary for specific applications,
     such as speech synthesis.

     The function `accept-process-output' returns non-`nil' if it did
     get some output, or `nil' if the timeout expired before output
     arrived.


File: elisp,  Node: Sentinels,  Next: Query Before Exit,  Prev: Output from Processes,  Up: Processes

37.10 Sentinels: Detecting Process Status Changes
=================================================

A "process sentinel" is a function that is called whenever the
associated process changes status for any reason, including signals
(whether sent by Emacs or caused by the process's own actions) that
terminate, stop, or continue the process.  The process sentinel is also
called if the process exits.  The sentinel receives two arguments: the
process for which the event occurred, and a string describing the type
of event.

   The string describing the event looks like one of the following:

   * `"finished\n"'.

   * `"exited abnormally with code EXITCODE\n"'.

   * `"NAME-OF-SIGNAL\n"'.

   * `"NAME-OF-SIGNAL (core dumped)\n"'.

   A sentinel runs only while Emacs is waiting (e.g., for terminal
input, or for time to elapse, or for process output).  This avoids the
timing errors that could result from running sentinels at random places
in the middle of other Lisp programs.  A program can wait, so that
sentinels will run, by calling `sit-for' or `sleep-for' (*note
Waiting::), or `accept-process-output' (*note Accepting Output::).
Emacs also allows sentinels to run when the command loop is reading
input.  `delete-process' calls the sentinel when it terminates a
running process.

   Emacs does not keep a queue of multiple reasons to call the sentinel
of one process; it records just the current status and the fact that
there has been a change.  Therefore two changes in status, coming in
quick succession, can call the sentinel just once.  However, process
termination will always run the sentinel exactly once.  This is because
the process status can't change again after termination.

   Emacs explicitly checks for output from the process before running
the process sentinel.  Once the sentinel runs due to process
termination, no further output can arrive from the process.

   A sentinel that writes the output into the buffer of the process
should check whether the buffer is still alive.  If it tries to insert
into a dead buffer, it will get an error.  If the buffer is dead,
`(buffer-name (process-buffer PROCESS))' returns `nil'.

   Quitting is normally inhibited within a sentinel--otherwise, the
effect of typing `C-g' at command level or to quit a user command would
be unpredictable.  If you want to permit quitting inside a sentinel,
bind `inhibit-quit' to `nil'.  In most cases, the right way to do this
is with the macro `with-local-quit'.  *Note Quitting::.

   If an error happens during execution of a sentinel, it is caught
automatically, so that it doesn't stop the execution of whatever
programs was running when the sentinel was started.  However, if
`debug-on-error' is non-`nil',  errors are not caught.  This makes it
possible to use the Lisp debugger to debug the sentinel.  *Note
Debugger::.

   While a sentinel is running, the process sentinel is temporarily set
to `nil' so that the sentinel won't run recursively.  For this reason
it is not possible for a sentinel to specify a new sentinel.

   Note that Emacs automatically saves and restores the match data
while executing sentinels.  *Note Match Data::.

 -- Function: set-process-sentinel process sentinel
     This function associates SENTINEL with PROCESS.  If SENTINEL is
     `nil', then the process will have no sentinel.  The default
     behavior when there is no sentinel is to insert a message in the
     process's buffer when the process status changes.

     Changes in process sentinels take effect immediately--if the
     sentinel is slated to be run but has not been called yet, and you
     specify a new sentinel, the eventual call to the sentinel will use
     the new one.

          (defun msg-me (process event)
             (princ
               (format "Process: %s had the event `%s'" process event)))
          (set-process-sentinel (get-process "shell") 'msg-me)
               => msg-me
          (kill-process (get-process "shell"))
               -| Process: #<process shell> had the event `killed'
               => #<process shell>

 -- Function: process-sentinel process
     This function returns the sentinel of PROCESS, or `nil' if it has
     none.

 -- Function: waiting-for-user-input-p
     While a sentinel or filter function is running, this function
     returns non-`nil' if Emacs was waiting for keyboard input from the
     user at the time the sentinel or filter function was called, or
     `nil' if it was not.


File: elisp,  Node: Query Before Exit,  Next: System Processes,  Prev: Sentinels,  Up: Processes

37.11 Querying Before Exit
==========================

When Emacs exits, it terminates all its subprocesses by sending them
the `SIGHUP' signal.  Because subprocesses may be doing valuable work,
Emacs normally asks the user to confirm that it is ok to terminate
them.  Each process has a query flag, which, if non-`nil', says that
Emacs should ask for confirmation before exiting and thus killing that
process.  The default for the query flag is `t', meaning _do_ query.

 -- Function: process-query-on-exit-flag process
     This returns the query flag of PROCESS.

 -- Function: set-process-query-on-exit-flag process flag
     This function sets the query flag of PROCESS to FLAG.  It returns
     FLAG.

     Here is an example of using `set-process-query-on-exit-flag' on a
     shell process to avoid querying:

          (set-process-query-on-exit-flag (get-process "shell") nil)
               => nil


File: elisp,  Node: System Processes,  Next: Transaction Queues,  Prev: Query Before Exit,  Up: Processes

37.12 Accessing Other Processes
===============================

In addition to accessing and manipulating processes that are
subprocesses of the current Emacs session, Emacs Lisp programs can also
access other processes running on the same machine.  We call these
"system processes", to distinguish them from Emacs subprocesses.

   Emacs provides several primitives for accessing system processes.
Not all platforms support these primitives; on those which don't, these
primitives return `nil'.

 -- Function: list-system-processes
     This function returns a list of all the processes running on the
     system.  Each process is identified by its PID, a numerical
     process ID that is assigned by the OS and distinguishes the process
     from all the other processes running on the same machine at the
     same time.

 -- Function: process-attributes pid
     This function returns an alist of attributes for the process
     specified by its process ID PID.  Each association in the alist is
     of the form `(KEY . VALUE)', where KEY designates the attribute
     and VALUE is the value of that attribute.  The various attribute
     KEYs that this function can return are listed below.  Not all
     platforms support all of these attributes; if an attribute is not
     supported, its association will not appear in the returned alist.
     Values that are numbers can be either integer or floating-point,
     depending on the magnitude of the value.

    `euid'
          The effective user ID of the user who invoked the process.
          The corresponding VALUE is a number.  If the process was
          invoked by the same user who runs the current Emacs session,
          the value is identical to what `user-uid' returns (*note User
          Identification::).

    `user'
          User name corresponding to the process's effective user ID, a
          string.

    `egid'
          The group ID of the effective user ID, a number.

    `group'
          Group name corresponding to the effective user's group ID, a
          string.

    `comm'
          The name of the command that runs in the process.  This is a
          string that usually specifies the name of the executable file
          of the process, without the leading directories.  However,
          some special system processes can report strings that do not
          correspond to an executable file of a program.

    `state'
          The state code of the process.  This is a short string that
          encodes the scheduling state of the process.  Here's a list
          of the most frequently seen codes:

         `"D"'
               uninterruptible sleep (usually I/O)

         `"R"'
               running

         `"S"'
               interruptible sleep (waiting for some event)

         `"T"'
               stopped, e.g., by a job control signal

         `"Z"'
               "zombie": a process that terminated, but was not reaped
               by its parent

          For the full list of the possible states, see the manual page
          of the `ps' command.

    `ppid'
          The process ID of the parent process, a number.

    `pgrp'
          The process group ID of the process, a number.

    `sess'
          The session ID of the process.  This is a number that is the
          process ID of the process's "session leader".

    `ttname'
          A string that is the name of the process's controlling
          terminal.  On Unix and GNU systems, this is normally the file
          name of the corresponding terminal device, such as
          `/dev/pts65'.

    `tpgid'
          The numerical process group ID of the foreground process
          group that uses the process's terminal.

    `minflt'
          The number of minor page faults caused by the process since
          its beginning.  (Minor page faults are those that don't
          involve reading from disk.)

    `majflt'
          The number of major page faults caused by the process since
          its beginning.  (Major page faults require a disk to be read,
          and are thus more expensive than minor page faults.)

    `cminflt'
    `cmajflt'
          Like `minflt' and `majflt', but include the number of page
          faults for all the child processes of the given process.

    `utime'
          Time spent by the process in the user context, for running the
          application's code.  The corresponding VALUE is in the
          `(HIGH LOW MICROSEC)' format, the same format used by
          functions `current-time' (*note current-time: Time of Day.)
          and `file-attributes' (*note File Attributes::).

    `stime'
          Time spent by the process in the system (kernel) context, for
          processing system calls.  The corresponding VALUE is in the
          same format as for `utime'.

    `time'
          The sum of `utime' and `stime'.  The corresponding VALUE is
          in the same format as for `utime'.

    `cutime'
    `cstime'
    `ctime'
          Like `utime', `stime', and `time', but include the times of
          all the child processes of the given process.

    `pri'
          The numerical priority of the process.

    `nice'
          The "nice value" of the process, a number.  (Processes with
          smaller nice values get scheduled more favorably.)

    `thcount'
          The number of threads in the process.

    `start'
          The time when the process was started, in the same
          `(HIGH LOW MICROSEC)' format used by `current-time' and by
          `file-attributes'.

    `etime'
          The time elapsed since the process started, in the
          `(HIGH LOW MICROSEC)' format.

    `vsize'
          The virtual memory size of the process, measured in kilobytes.

    `rss'
          The size of the process's "resident set", the number of
          kilobytes occupied by the process in the machine's physical
          memory.

    `pcpu'
          The percentage of the CPU time used by the process since it
          started.  The corresponding VALUE is a floating-point number
          between 0 and 100.

    `pmem'
          The percentage of the total physical memory installed on the
          machine used by the process's resident set.  The value is a
          floating-point number between 0 and 100.

    `args'
          The command-line with which the process was invoked.  This is
          a string in which individual command-line arguments are
          separated by blanks; whitespace characters that are embedded
          in the arguments are quoted as appropriate for the system's
          shell: escaped by backslash characters on GNU and Unix, and
          enclosed in double quote characters on Windows.  Thus, this
          command-line string can be directly used in primitives such
          as `shell-command'.



File: elisp,  Node: Transaction Queues,  Next: Network,  Prev: System Processes,  Up: Processes

37.13 Transaction Queues
========================

You can use a "transaction queue" to communicate with a subprocess
using transactions.  First use `tq-create' to create a transaction
queue communicating with a specified process.  Then you can call
`tq-enqueue' to send a transaction.

 -- Function: tq-create process
     This function creates and returns a transaction queue
     communicating with PROCESS.  The argument PROCESS should be a
     subprocess capable of sending and receiving streams of bytes.  It
     may be a child process, or it may be a TCP connection to a server,
     possibly on another machine.

 -- Function: tq-enqueue queue question regexp closure fn &optional
          delay-question
     This function sends a transaction to queue QUEUE.  Specifying the
     queue has the effect of specifying the subprocess to talk to.

     The argument QUESTION is the outgoing message that starts the
     transaction.  The argument FN is the function to call when the
     corresponding answer comes back; it is called with two arguments:
     CLOSURE, and the answer received.

     The argument REGEXP is a regular expression that should match text
     at the end of the entire answer, but nothing before; that's how
     `tq-enqueue' determines where the answer ends.

     If the argument DELAY-QUESTION is non-`nil', delay sending this
     question until the process has finished replying to any previous
     questions.  This produces more reliable results with some
     processes.

 -- Function: tq-close queue
     Shut down transaction queue QUEUE, waiting for all pending
     transactions to complete, and then terminate the connection or
     child process.

   Transaction queues are implemented by means of a filter function.
*Note Filter Functions::.


File: elisp,  Node: Network,  Next: Network Servers,  Prev: Transaction Queues,  Up: Processes

37.14 Network Connections
=========================

Emacs Lisp programs can open stream (TCP) and datagram (UDP) network
connections (*note Datagrams::) to other processes on the same machine
or other machines.  A network connection is handled by Lisp much like a
subprocess, and is represented by a process object.  However, the
process you are communicating with is not a child of the Emacs process,
has no process ID, and you can't kill it or send it signals.  All you
can do is send and receive data.  `delete-process' closes the
connection, but does not kill the program at the other end; that
program must decide what to do about closure of the connection.

   Lisp programs can listen for connections by creating network
servers.  A network server is also represented by a kind of process
object, but unlike a network connection, the network server never
transfers data itself.  When it receives a connection request, it
creates a new network connection to represent the connection just made.
(The network connection inherits certain information, including the
process plist, from the server.)  The network server then goes back to
listening for more connection requests.

   Network connections and servers are created by calling
`make-network-process' with an argument list consisting of
keyword/argument pairs, for example `:server t' to create a server
process, or `:type 'datagram' to create a datagram connection.  *Note
Low-Level Network::, for details.  You can also use the
`open-network-stream' function described below.

   To distinguish the different types of processes, the `process-type'
function returns the symbol `network' for a network connection or
server, `serial' for a serial port connection, or `real' for a real
subprocess.

   The `process-status' function returns `open', `closed', `connect',
or `failed' for network connections.  For a network server, the status
is always `listen'.  None of those values is possible for a real
subprocess.  *Note Process Information::.

   You can stop and resume operation of a network process by calling
`stop-process' and `continue-process'.  For a server process, being
stopped means not accepting new connections.  (Up to 5 connection
requests will be queued for when you resume the server; you can
increase this limit, unless it is imposed by the operating system--see
the `:server' keyword of `make-network-process', *note Network
Processes::.)  For a network stream connection, being stopped means not
processing input (any arriving input waits until you resume the
connection).  For a datagram connection, some number of packets may be
queued but input may be lost.  You can use the function
`process-command' to determine whether a network connection or server
is stopped; a non-`nil' value means yes.

   Emacs can create encrypted network connections, using either built-in
or external support.  The built-in support uses the GnuTLS ("Transport
Layer Security") library; see the GnuTLS project page
(http://www.gnu.org/software/gnutls/).  If your Emacs was compiled with
GnuTLS support, the function `gnutls-available-p' is defined and
returns non-`nil'.  For more details, *note Overview: (emacs-gnutls)Top.
The external support uses the `starttls.el' library, which requires a
helper utility such as `gnutls-cli' to be installed on the system.  The
`open-network-stream' function can transparently handle the details of
creating encrypted connections for you, using whatever support is
available.

 -- Function: open-network-stream name buffer host service &rest
          parameters
     This function opens a TCP connection, with optional encryption, and
     returns a process object that represents the connection.

     The NAME argument specifies the name for the process object.  It
     is modified as necessary to make it unique.

     The BUFFER argument is the buffer to associate with the
     connection.  Output from the connection is inserted in the buffer,
     unless you specify a filter function to handle the output.  If
     BUFFER is `nil', it means that the connection is not associated
     with any buffer.

     The arguments HOST and SERVICE specify where to connect to; HOST
     is the host name (a string), and SERVICE is the name of a defined
     network service (a string) or a port number (an integer).

     The remaining arguments PARAMETERS are keyword/argument pairs that
     are mainly relevant to encrypted connections:

    `:nowait BOOLEAN'
          If non-`nil', try to make an asynchronous connection.

    `:type TYPE'
          The type of connection.  Options are:

         `plain'
               An ordinary, unencrypted connection.

         `tls'
         `ssl'
               A TLS ("Transport Layer Security") connection.

         `nil'
         `network'
               Start with a plain connection, and if parameters
               `:success' and `:capability-command' are supplied, try
               to upgrade to an encrypted connection via STARTTLS.  If
               that fails, retain the unencrypted connection.

         `starttls'
               As for `nil', but if STARTTLS fails drop the connection.

         `shell'
               A shell connection.

    `:always-query-capabilities BOOLEAN'
          If non-`nil', always ask for the server's capabilities, even
          when doing a `plain' connection.

    `:capability-command CAPABILITY-COMMAND'
          Command string to query the host capabilities.

    `:end-of-command REGEXP'
    `:end-of-capability REGEXP'
          Regular expression matching the end of a command, or the end
          of the command CAPABILITY-COMMAND.  The latter defaults to
          the former.

    `:starttls-function FUNCTION'
          Function of one argument (the response to CAPABILITY-COMMAND),
          which returns either `nil', or the command to activate
          STARTTLS if supported.

    `:success REGEXP'
          Regular expression matching a successful STARTTLS negotiation.

    `:use-starttls-if-possible BOOLEAN'
          If non-`nil', do opportunistic STARTTLS upgrades even if Emacs
          doesn't have built-in TLS support.

    `:client-certificate LIST-OR-T'
          Either a list of the form `(KEY-FILE CERT-FILE)', naming the
          certificate key file and certificate file itself, or `t',
          meaning to query `auth-source' for this information (*note
          Overview: (auth)Top.).  Only used for TLS or STARTTLS.

    `:return-list CONS-OR-NIL'
          The return value of this function.  If omitted or `nil',
          return a process object.  Otherwise, a cons of the form
          `(PROCESS-OBJECT . PLIST)', where PLIST has keywords:

         `:greeting STRING-OR-NIL'
               If non-`nil', the greeting string returned by the host.

         `:capabilities STRING-OR-NIL'
               If non-`nil', the host's capability string.

         `:type SYMBOL'
               The connection type: `plain' or `tls'.




File: elisp,  Node: Network Servers,  Next: Datagrams,  Prev: Network,  Up: Processes

37.15 Network Servers
=====================

You create a server by calling `make-network-process' (*note Network
Processes::) with `:server t'.  The server will listen for connection
requests from clients.  When it accepts a client connection request,
that creates a new network connection, itself a process object, with
the following parameters:

   * The connection's process name is constructed by concatenating the
     server process's NAME with a client identification string.  The
     client identification string for an IPv4 connection looks like
     `<A.B.C.D:P>', which represents an address and port number.
     Otherwise, it is a unique number in brackets, as in `<NNN>'.  The
     number is unique for each connection in the Emacs session.

   * If the server's filter is non-`nil', the connection process does
     not get a separate process buffer; otherwise, Emacs creates a new
     buffer for the purpose.  The buffer name is the server's buffer
     name or process name, concatenated with the client identification
     string.

     The server's process buffer value is never used directly, but the
     log function can retrieve it and use it to log connections by
     inserting text there.

   * The communication type and the process filter and sentinel are
     inherited from those of the server.  The server never directly
     uses its filter and sentinel; their sole purpose is to initialize
     connections made to the server.

   * The connection's process contact information is set according to
     the client's addressing information (typically an IP address and a
     port number).  This information is associated with the
     `process-contact' keywords `:host', `:service', `:remote'.

   * The connection's local address is set up according to the port
     number used for the connection.

   * The client process's plist is initialized from the server's plist.


File: elisp,  Node: Datagrams,  Next: Low-Level Network,  Prev: Network Servers,  Up: Processes

37.16 Datagrams
===============

A "datagram" connection communicates with individual packets rather
than streams of data.  Each call to `process-send' sends one datagram
packet (*note Input to Processes::), and each datagram received results
in one call to the filter function.

   The datagram connection doesn't have to talk with the same remote
peer all the time.  It has a "remote peer address" which specifies
where to send datagrams to.  Each time an incoming datagram is passed
to the filter function, the peer address is set to the address that
datagram came from; that way, if the filter function sends a datagram,
it will go back to that place.  You can specify the remote peer address
when you create the datagram connection using the `:remote' keyword.
You can change it later on by calling `set-process-datagram-address'.

 -- Function: process-datagram-address process
     If PROCESS is a datagram connection or server, this function
     returns its remote peer address.

 -- Function: set-process-datagram-address process address
     If PROCESS is a datagram connection or server, this function sets
     its remote peer address to ADDRESS.


File: elisp,  Node: Low-Level Network,  Next: Misc Network,  Prev: Datagrams,  Up: Processes

37.17 Low-Level Network Access
==============================

You can also create network connections by operating at a lower level
than that of `open-network-stream', using `make-network-process'.

* Menu:

* Proc: Network Processes.   Using `make-network-process'.
* Options: Network Options.  Further control over network connections.
* Features: Network Feature Testing.
                             Determining which network features work on
                               the machine you are using.



Local Variables:
coding: iso-8859-1
End:
